-- Logs begin at Wed 2022-09-07 12:19:18 UTC, end at Wed 2022-09-07 13:35:49 UTC. -- Sep 07 12:50:05.577968 dv-cinder-853315-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 12:50:05.586723 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 12:50:05 2022] *** Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: nodename: dv-cinder-853315-7-iscsi Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: machine: x86_64 Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: clock source: unix Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: pcre jit disabled Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: detected number of CPU cores: 4 Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: current working directory: / Sep 07 12:50:05.589039 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: detected binary path: /usr/bin/uwsgi-core Sep 07 12:50:05.590130 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: your processes number limit is 47680 Sep 07 12:50:05.590130 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: your memory page size is 4096 bytes Sep 07 12:50:05.590130 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: detected max file descriptor number: 2048 Sep 07 12:50:05.590130 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: lock engine: pthread robust mutexes Sep 07 12:50:05.590130 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: thunder lock: enabled Sep 07 12:50:05.592235 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 12:50:05.592235 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 12:50:05.632154 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: Python main interpreter initialized at 0x55c3682399c0 Sep 07 12:50:05.632154 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: python threads support enabled Sep 07 12:50:05.632154 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: your server socket listen backlog is limited to 100 connections Sep 07 12:50:05.632154 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: your mercy for graceful operations on workers is 90 seconds Sep 07 12:50:05.632552 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 12:50:05.632552 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: *** Operational MODE: preforking *** Sep 07 12:50:05.632865 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: *** uWSGI is running in multiple interpreter mode *** Sep 07 12:50:05.632865 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: spawned uWSGI master process (pid: 81524) Sep 07 12:50:05.633046 dv-cinder-853315-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 12:50:05.634501 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: spawned uWSGI worker 1 (pid: 81536, cores: 1) Sep 07 12:50:05.634501 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: spawned uWSGI worker 2 (pid: 81537, cores: 1) Sep 07 12:50:05.634501 dv-cinder-853315-7-iscsi devstack@keystone.service[81524]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 12:50:09.995144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:50:09.995144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: from cryptography.utils import int_from_bytes Sep 07 12:50:09.995986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:50:09.995986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: from cryptography.utils import int_from_bytes Sep 07 12:50:10.001274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:50:10.001274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Sep 07 12:50:10.001885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:50:10.001885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Sep 07 12:50:10.046845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 12:50:10.047078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 12:50:10.047221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] command line args: [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 12:50:10.047500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 12:50:10.047636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ================================================================================ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 12:50:10.047909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] admin_token = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.048154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_dir = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.048316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.048521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_source = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.048699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] debug = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.048919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_config_append = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_dir = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.050812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.051325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.051325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.051485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.051701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 12:50:10.051815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.051900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 12:50:10.051980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.052059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.052130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] command line args: [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 12:50:10.052303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.052398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 12:50:10.052483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ================================================================================ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 12:50:10.052560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.052751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.052846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.052972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_dir = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_source = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] debug = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.053996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.054117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.054220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.054369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.054616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.054770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.054886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] publish_errors = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_config_append = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_dir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.055871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] standard_threads = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.056954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_journal = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_json = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_stderr = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_syslog = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.057944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.058024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.058134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.058211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.058292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.058364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.058545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.058633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.058795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.058876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.059017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.059269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.059351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.059487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] publish_errors = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.059635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.059817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.059897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.060071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.060310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.060545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.060723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.060855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.060971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.061085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.061203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.061315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.061434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.061547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.061658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] standard_threads = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.061795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.061907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.062017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.062214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.062214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.062440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.062552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_journal = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.062668 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.062786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_json = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.062904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.063017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_stderr = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.063017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_syslog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.063205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.063314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:50:10.063417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.063536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.063647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.063756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.063872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.063983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064210 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.064960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.065948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.066942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.067909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.068931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.069937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.070050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.070161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.070290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.070404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.070404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.070720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.070888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.071906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.072913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073109 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.073870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.074075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.074211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.074327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.074440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.074593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.074733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.074849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.075669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.075808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.075808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.076936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.077253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.077365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.077482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.077595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.077721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.077841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.077961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.078895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.079934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.080947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.081909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.082905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.083950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.external = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084862 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.084988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.password = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.token = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.085943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086176 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.086889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.087921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.088962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.089899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.external = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.090972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.password = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.token = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.091814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.092677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.093976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.094981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.095900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.096015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.096236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.096356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.096478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.096625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.096767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.096910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.097974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.098108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.098254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.098390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.098505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.098628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.098771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.098890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.099915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.100883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.101901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.102899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.103891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.104870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.105935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.106905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.107971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.108092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.108260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.108380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.108511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.108626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.108762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.108881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.109898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.110922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.111964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.112991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.113940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.114959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.115978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.116901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.117943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.118975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.119986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.120983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.121917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.122033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.122258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.122428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.122548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.122662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.122775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.122888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.123916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.124814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.125126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.125248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.125363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.125476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.125476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.125760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.125877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.126922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.127035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.127149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.127261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.127383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.127785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.127899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.128889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.129965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.130997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.131911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.132948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.133881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.134351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.135285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.135285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.135285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.135285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.135695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.135695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.135969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.136238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.136490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.136743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.136984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.137230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.137467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.137807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.138178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.139004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.139261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.139505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.139757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.139998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.140254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.140496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.140746 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.140980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.141223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.141461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:50:10.141635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 12:50:10.161991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.162223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.162422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.167371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.167607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.167835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.170945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.176999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.199416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.199659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.205025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.205265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.255356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.255899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.307153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.307380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.307789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.308029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.468431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.468680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.468861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.469073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.469286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.469480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.470341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.470577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.470801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.471029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.471242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.471463 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.493522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.493791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.493931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.494073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.494197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.494425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.494548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.494661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.494881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.495000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.495139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.495255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.495376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.495497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.507151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.507695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.520515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.520737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.520968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.521180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.521359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.521477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.521589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.521702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.521836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.521922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.522075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.522298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.522515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.522736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:50:10.539094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.539328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:50:10.539421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.539627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.539754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:50:10.539836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:50:10.539988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.540127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.540211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:50:10.540321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.540408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.540614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.540753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.540836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.540912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.541203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.550753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.551033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.551304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.551590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.551714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.551890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.552165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.552451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.558584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.559453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.561647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.561964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.562248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.562529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.562797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.563058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.563329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.563603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.563872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.564146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.564233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.564417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.564499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.564735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.565027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.565321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.565696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.565906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.566189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.566469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.566749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.567022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.587374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.588862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.589149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.590492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.592446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.592741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.594319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.594615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.594897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.597406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.597719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.598105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.601159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.601455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.604445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.604743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.607563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.607870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.608159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.608449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.608726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.609001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.609256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.609526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.611327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.611654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.611941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.612228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.612514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.612784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.613073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.613346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.625533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.625878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.630034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.630168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.630339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.633256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.633558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.634190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.637440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.637770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.637900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.640962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.642723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.644189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.644493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.644783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.645070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.645351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.645650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.645876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.649168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.649453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.649800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.650100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.650382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.650672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.661773 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.662161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.662433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.662694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.667095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.667395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.667670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.667912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.669648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.671196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.672700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.672978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.675246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.675691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.675950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.676250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.676533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.676888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.676993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.678441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.678719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.681321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.681692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.681994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.682275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.682669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.687988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.688276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.688555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.688834 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.689215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.689512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.689820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.694302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.694598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.694885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.695171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.695558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.695839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.696130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.705184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.708156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.710993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.711950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.712443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.713911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.714197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.714587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.714852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.718238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.719659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.720878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.721348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.721634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.722121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.722335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.725145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.725536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.725852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.726130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.728370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.729840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.733445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.733872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.733998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.734138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.734316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.734444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.734614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.741397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.741697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.741981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.742107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.742226 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.742441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.742604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.742692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.742771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.743054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.743338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.743614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.743883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.744150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.744448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.744775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.745042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.750467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:50:10.750771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.751053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.751333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.751610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.751913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.752177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.752462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.753038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.753316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.753592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.753939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.754240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.754513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:50:10.779830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c3682399c0 pid: 81536 (default app) Sep 07 12:50:10.783315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bee53b-d1d0-4af1-a4d8-d898628c5c41 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:10.783604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bee53b-d1d0-4af1-a4d8-d898628c5c41 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:10.783878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bee53b-d1d0-4af1-a4d8-d898628c5c41 None None] PATH_INFO: `/v3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:10.788201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Wed Sep 7 12:50:10 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:10.790231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c3682399c0 pid: 81537 (default app) Sep 07 12:50:17.746664 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98a2f-0069-4249-b7f8-b5587a2dd9d6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:17.747223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98a2f-0069-4249-b7f8-b5587a2dd9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:17.747223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c98a2f-0069-4249-b7f8-b5587a2dd9d6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:17.748703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:17.757586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:17.758341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:17.758341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:17.831050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: INFO dbcounter [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] Registered counter for database keystone Sep 07 12:50:17.851320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG oslo_db.sqlalchemy.engines [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81537) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 12:50:17.860378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writer thread running {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 12:50:18.056250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81537) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 12:50:18.056564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81537) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 12:50:18.117109 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:18.118781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:18.266364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 512 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 12:50:18.765948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98a2f-0069-4249-b7f8-b5587a2dd9d6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:18.837073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: INFO dbcounter [None req-f7c98a2f-0069-4249-b7f8-b5587a2dd9d6 None None] Registered counter for database keystone Sep 07 12:50:18.859509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG oslo_db.sqlalchemy.engines [None req-f7c98a2f-0069-4249-b7f8-b5587a2dd9d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81536) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 12:50:18.868482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writer thread running {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 12:50:18.990159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c98a2f-0069-4249-b7f8-b5587a2dd9d6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:18.995129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb42374-0855-4ec9-a4b7-16ca2febc437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:18.995778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb42374-0855-4ec9-a4b7-16ca2febc437 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:18.996060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb42374-0855-4ec9-a4b7-16ca2febc437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:18.996334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb42374-0855-4ec9-a4b7-16ca2febc437 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:19.008383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb42374-0855-4ec9-a4b7-16ca2febc437 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:19.061806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dcb42374-0855-4ec9-a4b7-16ca2febc437 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 12:50:19.062081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb42374-0855-4ec9-a4b7-16ca2febc437 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:19.072701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Sep 7 12:50:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 12:50:19.079175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:19.095003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6394df-f21c-4af5-9568-79eb1ca0731b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:19.100884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab8074-9dcb-4f79-8f2e-39a88ee403fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:19.102131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab8074-9dcb-4f79-8f2e-39a88ee403fb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:19.102500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab8074-9dcb-4f79-8f2e-39a88ee403fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:19.102600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab8074-9dcb-4f79-8f2e-39a88ee403fb admin admin] PATH_INFO: `/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:19.104242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab8074-9dcb-4f79-8f2e-39a88ee403fb admin admin] RBAC: Authorizing `identity:get_project(project_id=f6924cc44bdf49bd840f84494b6ffd4d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:19.157908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c8ab8074-9dcb-4f79-8f2e-39a88ee403fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 12:50:19.158242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab8074-9dcb-4f79-8f2e-39a88ee403fb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:19.162964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Sep 7 12:50:19 2022] GET /identity/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:21.357334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecfa2-9eb5-46d0-8052-986ac7297792 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:21.357990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecfa2-9eb5-46d0-8052-986ac7297792 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:21.357990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecfa2-9eb5-46d0-8052-986ac7297792 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:21.362085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:21.369013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4dfe7-d8c5-4433-9a38-a6feb881864c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:21.369547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4dfe7-d8c5-4433-9a38-a6feb881864c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:21.369547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4dfe7-d8c5-4433-9a38-a6feb881864c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:21.392048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-50b4dfe7-d8c5-4433-9a38-a6feb881864c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:21.393650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-50b4dfe7-d8c5-4433-9a38-a6feb881864c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:21.403531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 12:50:21.668017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ecfa2-9eb5-46d0-8052-986ac7297792 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:21.681254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ecfa2-9eb5-46d0-8052-986ac7297792 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:21.685905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18942cb2-099f-4f04-9eb6-f21b66afe32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:21.686500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18942cb2-099f-4f04-9eb6-f21b66afe32d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:21.686780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18942cb2-099f-4f04-9eb6-f21b66afe32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:21.687045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18942cb2-099f-4f04-9eb6-f21b66afe32d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:21.702549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18942cb2-099f-4f04-9eb6-f21b66afe32d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:21.704359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18942cb2-099f-4f04-9eb6-f21b66afe32d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:21.714147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Sep 7 12:50:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 12:50:21.720040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4dfe7-d8c5-4433-9a38-a6feb881864c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:21.729789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4dfe7-d8c5-4433-9a38-a6feb881864c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:21.735089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3b1e2-76c1-4fe7-a2b0-b544c969c32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:21.736698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3b1e2-76c1-4fe7-a2b0-b544c969c32b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:21.736698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3b1e2-76c1-4fe7-a2b0-b544c969c32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:21.736698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3b1e2-76c1-4fe7-a2b0-b544c969c32b admin admin] PATH_INFO: `/v3/users/ca1f33becd334f89a27a7404e04ed5aa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:21.738477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3b1e2-76c1-4fe7-a2b0-b544c969c32b admin admin] RBAC: Authorizing `identity:get_user(user_id=ca1f33becd334f89a27a7404e04ed5aa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:21.739758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3b1e2-76c1-4fe7-a2b0-b544c969c32b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:21.741380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Wed Sep 7 12:50:21 2022] GET /identity/v3/users/ca1f33becd334f89a27a7404e04ed5aa => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:24.050631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8ba5b-ab7d-402d-baec-41e52428069a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.051172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8ba5b-ab7d-402d-baec-41e52428069a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.051172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8ba5b-ab7d-402d-baec-41e52428069a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.052590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:24.058139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b522cc4-9c0d-4ebb-a413-3eb7a7c5ef03 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.058647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b522cc4-9c0d-4ebb-a413-3eb7a7c5ef03 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.058647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b522cc4-9c0d-4ebb-a413-3eb7a7c5ef03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.077931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1b522cc4-9c0d-4ebb-a413-3eb7a7c5ef03 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:24.079370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1b522cc4-9c0d-4ebb-a413-3eb7a7c5ef03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:24.115597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71628d3-435d-403c-aac5-d59a42b47ecf None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.116482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71628d3-435d-403c-aac5-d59a42b47ecf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.116482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71628d3-435d-403c-aac5-d59a42b47ecf None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.117458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 7/11] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:24.124477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.124939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.125273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.169865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG passlib.handlers.bcrypt [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81536) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 12:50:24.170313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG passlib.handlers.bcrypt [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81536) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 12:50:24.182537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 5/12] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:24.219777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:24.221410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:24.341349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 8/13] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:24.449767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b522cc4-9c0d-4ebb-a413-3eb7a7c5ef03 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:24.483458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:24.571688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b522cc4-9c0d-4ebb-a413-3eb7a7c5ef03 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:24.610663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:24.611859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:24.612662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.612929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.613220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.629961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:24.638845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:24.640280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:24.641274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 6/14] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:24.648093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:24.725965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fbe4-c000-462e-92dd-ff71c9fbdb9a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:24.729009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:24.741834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:24.761063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2fc2f-9f49-471a-8594-82872a296440 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:24.772106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:24.773283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.773561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.773863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.796576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:24.797192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.797492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.797781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.798453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:24.800307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:24.801899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:24.814779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 7/15] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:24.820636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:24.853284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:24.854899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:24.928641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8a16-f606-41e1-a8e6-de99d8db6ef7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:24.965262 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:24.966024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:24.966431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:24.966911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] PATH_INFO: `/v3/users/ca1f33becd334f89a27a7404e04ed5aa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:24.968453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1f33becd334f89a27a7404e04ed5aa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:24.969956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:24.970397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 9/16] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:50:24.971528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 8/17] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:24 2022] GET /identity/v3/users/ca1f33becd334f89a27a7404e04ed5aa => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:24.976056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:24.979075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:24.980203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:24.996279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:25.089861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9c27f-dd18-40ea-8d42-0cd3e3755dd4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:25.096102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d968f5-c7c5-42ac-9c6e-40d33842db45 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:25.135123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:25.135975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:25.136275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:25.136539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:25.136928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:25.137583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:25.137895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:25.138188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:25.139380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:25.141007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:25.142965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:25.148780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:25.158436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:25.160366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:50:25.179934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:25.180951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:50:25.182018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:50:25.182018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:50:25.182018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:25.182018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:50:25.187883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:25.243991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:25.250339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:25.265971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24aeabe-05bb-4128-ae3b-06ac5152e120 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:25.300842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c971a5-ec07-47f5-a482-46e852940a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:25.301460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c971a5-ec07-47f5-a482-46e852940a09 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:25.301762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c971a5-ec07-47f5-a482-46e852940a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:25.302034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c971a5-ec07-47f5-a482-46e852940a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:25.302762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c971a5-ec07-47f5-a482-46e852940a09 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.domain.id=default, user.id=ca1f33becd334f89a27a7404e04ed5aa) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:25.303102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c971a5-ec07-47f5-a482-46e852940a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:25.304125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5fe3f-9984-4e64-9f50-782e39ae6c4b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:25.304505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c971a5-ec07-47f5-a482-46e852940a09 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:25.311766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1322 bytes} [Wed Sep 7 12:50:25 2022] GET /identity/v3/role_assignments?user.id=ca1f33becd334f89a27a7404e04ed5aa&scope.domain.id=default&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:25.342088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a9bb4-2567-450d-97e5-63ff681e9363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:25.342762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a9bb4-2567-450d-97e5-63ff681e9363 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:25.343046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a9bb4-2567-450d-97e5-63ff681e9363 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:25.343353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a9bb4-2567-450d-97e5-63ff681e9363 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:25.343969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a9bb4-2567-450d-97e5-63ff681e9363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:25.344290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a9bb4-2567-450d-97e5-63ff681e9363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:25.345771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a9bb4-2567-450d-97e5-63ff681e9363 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:25.367086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:28.271287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8f5fd-5244-4537-9397-3a4e87acb51f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.271829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8f5fd-5244-4537-9397-3a4e87acb51f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.271829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8f5fd-5244-4537-9397-3a4e87acb51f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.273603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:28.279789 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcacdde-3573-4d66-9ce3-526e32998423 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.280367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcacdde-3573-4d66-9ce3-526e32998423 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.280367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcacdde-3573-4d66-9ce3-526e32998423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.311147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7dcacdde-3573-4d66-9ce3-526e32998423 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:28.312439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7dcacdde-3573-4d66-9ce3-526e32998423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:28.391025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e2db-6e96-4597-b341-4bed3cb62768 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.391331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e2db-6e96-4597-b341-4bed3cb62768 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.391575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e2db-6e96-4597-b341-4bed3cb62768 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.392911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:50:28.399731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd0889-cdec-4859-85c3-6c0c87b00f44 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.400339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd0889-cdec-4859-85c3-6c0c87b00f44 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.400854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd0889-cdec-4859-85c3-6c0c87b00f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.427347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:28.437594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-98cd0889-cdec-4859-85c3-6c0c87b00f44 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:28.438825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98cd0889-cdec-4859-85c3-6c0c87b00f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:28.449559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e0f79-1cab-441c-b991-3cdda42133b0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.449891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e0f79-1cab-441c-b991-3cdda42133b0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.450140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e0f79-1cab-441c-b991-3cdda42133b0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.451938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 13/25] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:28.464447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bedaea-0d8f-435c-8b00-573037610fd8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.464738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bedaea-0d8f-435c-8b00-573037610fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.465001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bedaea-0d8f-435c-8b00-573037610fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.490146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-61bedaea-0d8f-435c-8b00-573037610fd8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:28.491809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-61bedaea-0d8f-435c-8b00-573037610fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:28.572654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 13/26] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:28.587867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:28.620530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 14/27] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:28.826759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abca300-a72a-42d9-a4e6-9c724f0616d3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.827081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abca300-a72a-42d9-a4e6-9c724f0616d3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.827966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abca300-a72a-42d9-a4e6-9c724f0616d3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.829520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 14/28] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:28.847751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c336c25-7726-4821-8888-36777b2b872e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:28.848050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c336c25-7726-4821-8888-36777b2b872e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:28.848319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c336c25-7726-4821-8888-36777b2b872e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:28.871738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3c336c25-7726-4821-8888-36777b2b872e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:28.872945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3c336c25-7726-4821-8888-36777b2b872e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:28.888807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abca300-a72a-42d9-a4e6-9c724f0616d3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:29.037523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 15/29] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:29.041065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c336c25-7726-4821-8888-36777b2b872e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:29.059213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:29.119433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abca300-a72a-42d9-a4e6-9c724f0616d3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:29.198444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:29.199114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:29.199400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:29.199680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:29.209967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c336c25-7726-4821-8888-36777b2b872e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:29.222889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:29.224474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:29.225968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:50:29.246265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:29.247238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:50:29.248062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:50:29.248062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:50:29.248062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:29.250658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 15/30] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:50:29.251775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:29.316751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:29.317756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:29.318217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:29.318666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:29.352636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:29.359462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:29.360389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:29.361263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:29.361946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:29.391854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe078408-d7fc-4137-9fe8-84a19888c9fc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:29.434253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:29.448178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:29.449403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:29.450159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:29.451021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:29.452236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:29.454592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:29.516659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f247c1-3b57-4474-9071-864a675545fb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:29.556265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:29.557401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:29.557898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:29.558489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:29.559373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:29.560000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:29.562564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:29.575715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 17/32] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:29.576846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:29.593654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 16/33] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 12:50:29 2022] POST /identity/v3/roles => generated 219 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:50:29.602426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:29.619904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:29.728514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561e025-b58d-4a6e-96df-35503d6accc4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:29.808685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:29.809345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:29.810490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:29.810490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:29.810687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:29.812324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:29.864094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37046633-483e-4cb3-a830-561b3978ee23 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:29.908557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:29.909343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:29.909632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:29.909977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:29.910851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:29.911158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:29.912821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:29.923232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 17/34] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:29.924162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:29.980781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:30.006527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 18/35] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 12:50:29 2022] POST /identity/v3/roles => generated 225 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:50:30.007882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:30.043684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec409b7d-83a0-4702-80f3-139652bd7e84 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:30.084381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fb7ec-d3a2-43a7-aec2-bc9152cec85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:30.085118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fb7ec-d3a2-43a7-aec2-bc9152cec85c None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.085409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fb7ec-d3a2-43a7-aec2-bc9152cec85c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.085711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fb7ec-d3a2-43a7-aec2-bc9152cec85c None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.086391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fb7ec-d3a2-43a7-aec2-bc9152cec85c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:30.088128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fb7ec-d3a2-43a7-aec2-bc9152cec85c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:30.123870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:30.130146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 18/36] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:50:29 2022] POST /identity/v3/projects => generated 301 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:50:30.132946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fa11f-df2a-4f66-975e-ef5c20d05b5d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.133509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fa11f-df2a-4f66-975e-ef5c20d05b5d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.133826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fa11f-df2a-4f66-975e-ef5c20d05b5d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.135396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 19/37] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:30.143258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eea98-51a3-40e5-a73e-0aa7891f8c57 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.143598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eea98-51a3-40e5-a73e-0aa7891f8c57 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.143814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eea98-51a3-40e5-a73e-0aa7891f8c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.157327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78ef92-f21f-45a5-b7d7-56789dcb20ae None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:30.171698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bf5eea98-51a3-40e5-a73e-0aa7891f8c57 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:30.173622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bf5eea98-51a3-40e5-a73e-0aa7891f8c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:30.221091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:30.222664 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.223182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.223738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] PATH_INFO: `/v3/users/ca1f33becd334f89a27a7404e04ed5aa` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.228178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1f33becd334f89a27a7404e04ed5aa)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:30.231564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:30.235173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 19/38] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:30 2022] GET /identity/v3/users/ca1f33becd334f89a27a7404e04ed5aa => generated 247 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:30.244359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:30.267318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 20/39] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:30.332259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0a20c-d093-4ddc-968d-656acaae9a77 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.332595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0a20c-d093-4ddc-968d-656acaae9a77 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.332826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0a20c-d093-4ddc-968d-656acaae9a77 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.335721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 21/40] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:30.340158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f560475-d617-4975-93ea-a0603da6b901 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.340456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f560475-d617-4975-93ea-a0603da6b901 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.340801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f560475-d617-4975-93ea-a0603da6b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.373361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8f560475-d617-4975-93ea-a0603da6b901 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:30.375031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8f560475-d617-4975-93ea-a0603da6b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:30.417948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9acd2-db33-4927-81dc-300a66afdab9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:30.419809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:30.472364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:30.473036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.473323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.473602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.474970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:30.476511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:30.478194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 20/41] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:30.485092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:30.492204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 22/42] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:30.531028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:30.584988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f560475-d617-4975-93ea-a0603da6b901 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:30.645038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718db1bf-1cf4-4bb7-9dd9-d43e2a912966 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:30.690151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:30.690857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.691141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.691422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] PATH_INFO: `/v3/domains/default/users/ca1f33becd334f89a27a7404e04ed5aa/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.694799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ca1f33becd334f89a27a7404e04ed5aa, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:30.696953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:30.716706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 21/43] 10.222.0.17 () {58 vars in 1361 bytes} [Wed Sep 7 12:50:30 2022] PUT /identity/v3/domains/default/users/ca1f33becd334f89a27a7404e04ed5aa/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:50:30.767689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:30.831192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f560475-d617-4975-93ea-a0603da6b901 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:30.883206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:30.885065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:30.886101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.886577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.886963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.887898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:30.890396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:30.910830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 23/44] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 12:50:30 2022] POST /identity/v3/roles => generated 223 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:50:30.913178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c61d4e-fadb-4ae5-9cd4-98fa68d1ad14 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:30.957619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219bed1-5e28-4f7d-940c-455bc017ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:30.958330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219bed1-5e28-4f7d-940c-455bc017ca40 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:30.958613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219bed1-5e28-4f7d-940c-455bc017ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:30.958889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219bed1-5e28-4f7d-940c-455bc017ca40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:30.960209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219bed1-5e28-4f7d-940c-455bc017ca40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:30.961760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219bed1-5e28-4f7d-940c-455bc017ca40 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:30.963412 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 22/45] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:30.969509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:31.000225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:31.115339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0c060-e996-404c-8d33-18ce7b375da0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:31.153012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc16e9-3339-438b-b1f7-c368f102c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:31.153645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc16e9-3339-438b-b1f7-c368f102c8dd None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:31.153943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc16e9-3339-438b-b1f7-c368f102c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:31.154215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc16e9-3339-438b-b1f7-c368f102c8dd None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:31.154881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc16e9-3339-438b-b1f7-c368f102c8dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:31.156674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc16e9-3339-438b-b1f7-c368f102c8dd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:31.188219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:50:30 2022] POST /identity/v3/projects => generated 312 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:50:32.936952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb93a9-60d3-4ce7-9876-851e0717026f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:32.937730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb93a9-60d3-4ce7-9876-851e0717026f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:32.937730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb93a9-60d3-4ce7-9876-851e0717026f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:32.938916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:32.944624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bac0a-98b1-432f-ae4c-9edce5e1abcc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:32.945133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bac0a-98b1-432f-ae4c-9edce5e1abcc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:32.945218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bac0a-98b1-432f-ae4c-9edce5e1abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:32.964268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ae4bac0a-98b1-432f-ae4c-9edce5e1abcc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:32.965484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae4bac0a-98b1-432f-ae4c-9edce5e1abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:33.047689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:33.317319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb93a9-60d3-4ce7-9876-851e0717026f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:33.426157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf13c-569a-4c3d-b996-87e23ac20ecf None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:33.426426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf13c-569a-4c3d-b996-87e23ac20ecf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:33.426868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf13c-569a-4c3d-b996-87e23ac20ecf None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:33.428224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 26/49] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:33.433637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0cfca-3ad0-44e8-9894-e067f828a297 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:33.433956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0cfca-3ad0-44e8-9894-e067f828a297 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:33.434266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0cfca-3ad0-44e8-9894-e067f828a297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:33.441161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:33.444051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb93a9-60d3-4ce7-9876-851e0717026f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:33.463288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-20b0cfca-3ad0-44e8-9894-e067f828a297 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:33.464515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-20b0cfca-3ad0-44e8-9894-e067f828a297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:33.486996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:33.487651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:33.487939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:33.488238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:33.499778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:33.504699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:33.506174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:33.507022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 24/50] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:50:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:33.512231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:33.526068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:33.542773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 27/51] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:33.630454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b04502-e66d-49a5-93fb-f73aff03a6ec None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:33.674009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:33.674635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:33.674898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:33.675172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:33.675837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:33.676139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:33.677533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:33.691840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 25/52] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:33.698228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0cfca-3ad0-44e8-9894-e067f828a297 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:33.814138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:33.817738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0cfca-3ad0-44e8-9894-e067f828a297 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:33.856358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:33.864121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:33.864926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:33.865258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:33.865589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] PATH_INFO: `/v3/users/ca1f33becd334f89a27a7404e04ed5aa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:33.867208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1f33becd334f89a27a7404e04ed5aa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:33.868967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:33.870945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 28/53] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:33 2022] GET /identity/v3/users/ca1f33becd334f89a27a7404e04ed5aa => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:33.877538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:33.896924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:33.929311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5487e3-04a9-4964-9e7c-8826349b7fd8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:34.002945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1872b-8a8d-487e-b243-d8505f7bcee5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:34.049442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:34.050156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:34.050455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:34.050741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:34.052074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:34.054170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:34.055505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 29/54] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:34.062230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:34.078452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:34.079142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:34.079447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:34.079721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:34.080868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:34.082386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:34.083952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 26/55] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:34.089903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:34.138449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:34.175520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a2a2b-e3ea-4c50-b79b-62618ec0c8cc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:34.205464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23d185-0c1e-4c6b-a22c-143ad08abc46 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:34.214416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535184d-49a8-4be9-a58d-55041ae24731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:34.215122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535184d-49a8-4be9-a58d-55041ae24731 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:34.215554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535184d-49a8-4be9-a58d-55041ae24731 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:34.215904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535184d-49a8-4be9-a58d-55041ae24731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:34.219678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535184d-49a8-4be9-a58d-55041ae24731 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.domain.id=default, user.id=ca1f33becd334f89a27a7404e04ed5aa) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:34.219678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535184d-49a8-4be9-a58d-55041ae24731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:34.219678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535184d-49a8-4be9-a58d-55041ae24731 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:34.226304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 30/56] 10.222.0.17 () {56 vars in 1322 bytes} [Wed Sep 7 12:50:34 2022] GET /identity/v3/role_assignments?user.id=ca1f33becd334f89a27a7404e04ed5aa&scope.domain.id=default&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 530 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:34.248569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dfe4d-25fb-45c2-8123-d0ab63d524e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:34.249335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfe4d-25fb-45c2-8123-d0ab63d524e9 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:34.249864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfe4d-25fb-45c2-8123-d0ab63d524e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:34.250298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dfe4d-25fb-45c2-8123-d0ab63d524e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:34.251164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dfe4d-25fb-45c2-8123-d0ab63d524e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:34.253620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dfe4d-25fb-45c2-8123-d0ab63d524e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:34.274340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:34.297960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 27/57] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Sep 7 12:50:34 2022] POST /identity/v3/projects => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:50:36.593522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7eefe-13b0-494a-8a5f-3c5cdea88355 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:36.594277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7eefe-13b0-494a-8a5f-3c5cdea88355 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:36.594277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7eefe-13b0-494a-8a5f-3c5cdea88355 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:36.595939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 31/58] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:36.602819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e564-f07d-49e9-92d9-39923a2d8497 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:36.603626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e564-f07d-49e9-92d9-39923a2d8497 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:36.603768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e564-f07d-49e9-92d9-39923a2d8497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:36.629227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8c74e564-f07d-49e9-92d9-39923a2d8497 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:36.631189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8c74e564-f07d-49e9-92d9-39923a2d8497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:36.713717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 28/59] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:36.994899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7eefe-13b0-494a-8a5f-3c5cdea88355 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:37.112406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7eefe-13b0-494a-8a5f-3c5cdea88355 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:37.145566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:37.155466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ba95a-eeac-4a40-8bd6-9836842b2421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:37.156126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ba95a-eeac-4a40-8bd6-9836842b2421 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:37.156413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ba95a-eeac-4a40-8bd6-9836842b2421 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:37.156693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ba95a-eeac-4a40-8bd6-9836842b2421 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:37.157899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ba95a-eeac-4a40-8bd6-9836842b2421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:37.159377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ba95a-eeac-4a40-8bd6-9836842b2421 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:37.160976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 32/60] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:37.166730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74e564-f07d-49e9-92d9-39923a2d8497 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:37.295150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74e564-f07d-49e9-92d9-39923a2d8497 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:37.324275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:37.337936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556e5e9-a4b6-4690-8377-4a3d7cf29e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:37.338565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556e5e9-a4b6-4690-8377-4a3d7cf29e97 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:37.338854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556e5e9-a4b6-4690-8377-4a3d7cf29e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:37.339130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556e5e9-a4b6-4690-8377-4a3d7cf29e97 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:37.339793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556e5e9-a4b6-4690-8377-4a3d7cf29e97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:37.341228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556e5e9-a4b6-4690-8377-4a3d7cf29e97 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:37.371373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 29/61] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:50:37 2022] POST /identity/v3/users => generated 275 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:50:40.967315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee655-4620-4270-99ed-6f5ea68c13ae None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:40.968677 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee655-4620-4270-99ed-6f5ea68c13ae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:40.969254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee655-4620-4270-99ed-6f5ea68c13ae None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:40.971309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 33/62] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:40.993482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831628f3-13bd-40a0-9f7c-d6b77538a2ee None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:40.995967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831628f3-13bd-40a0-9f7c-d6b77538a2ee None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:40.996449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831628f3-13bd-40a0-9f7c-d6b77538a2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.040835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-831628f3-13bd-40a0-9f7c-d6b77538a2ee None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:41.043316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-831628f3-13bd-40a0-9f7c-d6b77538a2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:41.169122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 30/63] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:41.234307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2a255-568e-4224-9f11-a1b7f3fcd981 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:41.234914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2a255-568e-4224-9f11-a1b7f3fcd981 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:41.235416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2a255-568e-4224-9f11-a1b7f3fcd981 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.237207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 34/64] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:41.243360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587adf9-dd49-43b6-80cc-61f6467e5cba None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:41.243986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587adf9-dd49-43b6-80cc-61f6467e5cba None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:41.244459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587adf9-dd49-43b6-80cc-61f6467e5cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.272173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9587adf9-dd49-43b6-80cc-61f6467e5cba None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:41.273844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9587adf9-dd49-43b6-80cc-61f6467e5cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:41.383575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 31/65] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:41.453371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcbb60-7765-416c-b45b-0cb795fe4281 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:41.454286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcbb60-7765-416c-b45b-0cb795fe4281 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:41.454781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcbb60-7765-416c-b45b-0cb795fe4281 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.456357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 35/66] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:41.462291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e77ef5-5f90-49f8-80c1-4beb451865c0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:41.462585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e77ef5-5f90-49f8-80c1-4beb451865c0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:41.463051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e77ef5-5f90-49f8-80c1-4beb451865c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.490915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-97e77ef5-5f90-49f8-80c1-4beb451865c0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:41.492408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-97e77ef5-5f90-49f8-80c1-4beb451865c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:41.501445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcbb60-7765-416c-b45b-0cb795fe4281 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:41.528704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:41.584357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 32/67] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:41.711400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e77ef5-5f90-49f8-80c1-4beb451865c0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:41.724464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcbb60-7765-416c-b45b-0cb795fe4281 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:41.770471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:41.771097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:41.771383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:41.771664 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.790187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:41.800066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:41.801044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:41.803092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 36/68] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:50:41.810032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:41.864753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e77ef5-5f90-49f8-80c1-4beb451865c0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:41.898661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:41.919602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7402a-e067-4de8-8f46-b1a49d1e5828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:41.920525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7402a-e067-4de8-8f46-b1a49d1e5828 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:41.921020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7402a-e067-4de8-8f46-b1a49d1e5828 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:41.921438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7402a-e067-4de8-8f46-b1a49d1e5828 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.935715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7402a-e067-4de8-8f46-b1a49d1e5828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-72e7402a-e067-4de8-8f46-b1a49d1e5828 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:41.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:41.942647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:41.943425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 33/69] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:50:41.945184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fec114-a80c-475d-8556-33b92f9de5fd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:41.945184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fec114-a80c-475d-8556-33b92f9de5fd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:41.945441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fec114-a80c-475d-8556-33b92f9de5fd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:41.947044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 34/70] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:41.947677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fec114-a80c-475d-8556-33b92f9de5fd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:41.966310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402f82-d1e2-4b37-96f0-d58fa12614a4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:41.980799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:42.004691 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.005356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.005903 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.006686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.007840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:42.008252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:42.009707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:42.019411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 37/71] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:42.022733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.061033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fec114-a80c-475d-8556-33b92f9de5fd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.097851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e46883-f94b-4a47-a90f-04ea963de495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.098511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e46883-f94b-4a47-a90f-04ea963de495 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.098786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e46883-f94b-4a47-a90f-04ea963de495 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.099057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e46883-f94b-4a47-a90f-04ea963de495 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.110792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e46883-f94b-4a47-a90f-04ea963de495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-57e46883-f94b-4a47-a90f-04ea963de495 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:42.115859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:42.116921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:42.117770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 35/72] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:50:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:42.118423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03b12-05f4-4992-bc97-61cce4fad7f4 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.118800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03b12-05f4-4992-bc97-61cce4fad7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.119069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad03b12-05f4-4992-bc97-61cce4fad7f4 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.125583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 36/73] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:42.125583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53447fb9-c506-48cc-a68f-39e6da6aa3e5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.125583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53447fb9-c506-48cc-a68f-39e6da6aa3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.125583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53447fb9-c506-48cc-a68f-39e6da6aa3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.143816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-53447fb9-c506-48cc-a68f-39e6da6aa3e5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:42.145198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-53447fb9-c506-48cc-a68f-39e6da6aa3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:42.152800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81992365-4feb-4d32-af99-90ccc3f7fb5f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.181705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:42.194573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.195620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.195910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.196186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.197196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:42.197502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:42.198990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:42.208733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 38/74] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:42.209651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.223564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 37/75] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:42.226660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53447fb9-c506-48cc-a68f-39e6da6aa3e5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.248218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:42.332183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81a29-bb1b-4ae9-a689-e0fa76dfeddc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.364162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53447fb9-c506-48cc-a68f-39e6da6aa3e5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.369511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443397d-50ff-4918-a8b9-5e7d87304d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.370219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443397d-50ff-4918-a8b9-5e7d87304d4f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.370501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443397d-50ff-4918-a8b9-5e7d87304d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.370777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443397d-50ff-4918-a8b9-5e7d87304d4f None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.388618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443397d-50ff-4918-a8b9-5e7d87304d4f None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:42.390297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443397d-50ff-4918-a8b9-5e7d87304d4f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:42.392611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:42.393962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3cd89-28b1-4db0-b99b-61e9f5a9d355 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.394449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3cd89-28b1-4db0-b99b-61e9f5a9d355 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.394790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3cd89-28b1-4db0-b99b-61e9f5a9d355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.416815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f8f3cd89-28b1-4db0-b99b-61e9f5a9d355 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:42.418287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f8f3cd89-28b1-4db0-b99b-61e9f5a9d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:42.434799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.435586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.435874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.436148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.436847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:42.437136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:42.439001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:42.448505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 38/77] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:42.454766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.463999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:42.503327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 40/78] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:42.504206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3cd89-28b1-4db0-b99b-61e9f5a9d355 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.564016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:42.592961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7143f7-1cfc-4563-a4a7-fa7e942ec22f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.639976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.640657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.641063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.641489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] PATH_INFO: `/v3/users/ca1f33becd334f89a27a7404e04ed5aa` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.643485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] RBAC: Authorizing `identity:get_user(user_id=ca1f33becd334f89a27a7404e04ed5aa)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:42.644950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:42.646065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3cd89-28b1-4db0-b99b-61e9f5a9d355 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.646822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/users/ca1f33becd334f89a27a7404e04ed5aa => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:42.647485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.695436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.696751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.697458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.698048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.748476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:42.770844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3721469-d017-46b7-a414-dde8c4034f1c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.780598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:42.782193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:42.818411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371462c-cf63-4758-b057-750819064050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.819085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371462c-cf63-4758-b057-750819064050 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.819380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371462c-cf63-4758-b057-750819064050 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.819653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371462c-cf63-4758-b057-750819064050 None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.820832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371462c-cf63-4758-b057-750819064050 None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:42.823469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371462c-cf63-4758-b057-750819064050 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:42.824234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 40/80] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:42.824756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371462c-cf63-4758-b057-750819064050 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.848270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:42.901681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 41/81] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:50:42.907434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:42.946014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371462c-cf63-4758-b057-750819064050 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:42.985652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:42.986321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:42.986611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:42.986930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:42.990345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.000651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:43.006185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:43.010182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 41/82] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:43.010995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.029357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4456a-c724-49fd-95d1-48cf98e3a777 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.065212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.065858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.066145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.066421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.104903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.135223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0caee-0887-4c38-8563-08776145fb97 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.150172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.151684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.170732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.171659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.171972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.172311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.174169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.175756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.177709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 42/83] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:43.178231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.244245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.286512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 42/84] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:50:43.290786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.300808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc8ee3-3ab3-4b30-98ef-a404b44e6cc3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.341389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.342041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.342329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.342609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.358298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.414162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307fc83-666c-4919-ad10-454f7e686d91 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.427724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.429515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.455623 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.456545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.456825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.457103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.458325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.459796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.461476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 43/85] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:43.462299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.509213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.541798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 43/86] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:50:43.542616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.589073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d41d07-3025-4234-b880-a4291efb8d80 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.598648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.625572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.626140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.626520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.626998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.628099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:43.628589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.630947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.643828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 44/87] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:43.644347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.668846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa139732-beae-49ac-b790-1edb72383f6e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.710037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.710677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.710958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.711232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.711944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.713450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.738639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 44/88] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:50:43 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:50:43.739484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.779675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10bdc4-abea-461c-8c35-7aa90a18fd6a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.781869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.827695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.828770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.829061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.829354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.831358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.832979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.839886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 45/89] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:43.839886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.881181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:43.886089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33738d-7114-4002-b95d-a553038cb63f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:43.934648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:43.936152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:43.936152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:43.936152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:43.936756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=08b02c5702264b2d9c30d73839dd83ba) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:43.937542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:43.938823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:43.953737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 45/90] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/role_assignments?user.id=08b02c5702264b2d9c30d73839dd83ba&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:43.953737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:43.988756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914afaf1-681a-49a8-b1f3-6b6f52fccc42 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:44.038893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:44.039554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:44.040005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:44.040238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:44.041442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:44.043021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:44.045025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 46/91] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:44.052288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:44.128604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d6d73-029a-4db6-beca-1998a0b2f48f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:44.134964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:44.212717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:44.214300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:44.215504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:44.216013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:44.218628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:44.220420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:44.222840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 46/92] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:43 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:44.223851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:44.239716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1d34e-a437-4f4b-b520-406fb13fd45c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:44.268084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:44.295287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:44.296766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:44.297394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:44.297930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:44.298909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=ca1f33becd334f89a27a7404e04ed5aa) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:44.299417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:44.301086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:44.308395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 47/93] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:50:44 2022] GET /identity/v3/role_assignments?user.id=ca1f33becd334f89a27a7404e04ed5aa&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:44.309653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:44.363358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3bc45-61b1-47cb-83d5-e384aca1c27a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:44.431367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:44.432016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:44.432297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:44.432572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:44.433307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] RBAC: Adding query filter params (role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=08b02c5702264b2d9c30d73839dd83ba) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:44.433619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:44.435178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:44.447417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 47/94] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:50:44 2022] GET /identity/v3/role_assignments?user.id=08b02c5702264b2d9c30d73839dd83ba&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:44.465368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d001a3e-cbd3-4ace-92ec-4b4b85d70b48 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:44.475778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:44.510922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:44.511894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:44.512523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:44.512886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:44.627828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:44.629640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:44.781688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 48/95] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:50:44.787895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:44.921082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:44.929351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803932-dc4d-40fb-821f-aa47cf332882 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:44.977177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5d353-726f-4612-b768-9edd32396ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:44.977867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d353-726f-4612-b768-9edd32396ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:44.978160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d353-726f-4612-b768-9edd32396ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:44.978437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d353-726f-4612-b768-9edd32396ca0 None admin] PATH_INFO: `/v3/projects/5dc2de4368bd4ce092cfff9965cd7979` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:44.982433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5d353-726f-4612-b768-9edd32396ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=5dc2de4368bd4ce092cfff9965cd7979)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:44.983945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5d353-726f-4612-b768-9edd32396ca0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:44.985945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 48/96] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:44 2022] GET /identity/v3/projects/5dc2de4368bd4ce092cfff9965cd7979 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:44.992211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:44.994718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:45.119427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08a2c4-4d5e-43d6-b4d9-e0ad049c9c0b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:45.175803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ad19-98cb-4d6f-8823-a0b644f65471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:45.176441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ad19-98cb-4d6f-8823-a0b644f65471 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:45.176725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ad19-98cb-4d6f-8823-a0b644f65471 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:45.177002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ad19-98cb-4d6f-8823-a0b644f65471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:45.177772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0ad19-98cb-4d6f-8823-a0b644f65471 None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=5dc2de4368bd4ce092cfff9965cd7979, user.id=08b02c5702264b2d9c30d73839dd83ba) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:45.178085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0ad19-98cb-4d6f-8823-a0b644f65471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:45.179578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0ad19-98cb-4d6f-8823-a0b644f65471 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:45.190540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 49/97] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:50:44 2022] GET /identity/v3/role_assignments?user.id=08b02c5702264b2d9c30d73839dd83ba&scope.project.id=5dc2de4368bd4ce092cfff9965cd7979&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:47.641323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c75338-afd3-4115-aea0-6258b7e291a5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.641861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c75338-afd3-4115-aea0-6258b7e291a5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.641861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c75338-afd3-4115-aea0-6258b7e291a5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.643328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 49/98] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:47.650195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f040a-993a-4687-b82d-d21aabe59ea2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.650711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f040a-993a-4687-b82d-d21aabe59ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.650783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f040a-993a-4687-b82d-d21aabe59ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.678932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3c1f040a-993a-4687-b82d-d21aabe59ea2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:47.680271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c1f040a-993a-4687-b82d-d21aabe59ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:47.739840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b27b4-88fd-40b6-863c-f2c448bc320b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.740219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b27b4-88fd-40b6-863c-f2c448bc320b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.740578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b27b4-88fd-40b6-863c-f2c448bc320b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.741990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 50/99] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:47.752813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fafd63-ee93-48a7-a849-db332dd74a80 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.753127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fafd63-ee93-48a7-a849-db332dd74a80 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.753379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fafd63-ee93-48a7-a849-db332dd74a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.792001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b7fafd63-ee93-48a7-a849-db332dd74a80 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:47.793243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b7fafd63-ee93-48a7-a849-db332dd74a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:47.811863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:47.823862 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 50/100] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:47.825208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc07db8-0d2e-484f-a47b-e4f2914ee3ae None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.825501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc07db8-0d2e-484f-a47b-e4f2914ee3ae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.825807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc07db8-0d2e-484f-a47b-e4f2914ee3ae None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.836689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 51/101] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:47.851998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463653f8-70a0-4228-82ab-ba2aad640d23 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.852518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463653f8-70a0-4228-82ab-ba2aad640d23 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.853020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463653f8-70a0-4228-82ab-ba2aad640d23 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.855629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 52/102] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:47.862502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ebdf-ae3f-4139-8caf-e41e224fba99 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.863004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ebdf-ae3f-4139-8caf-e41e224fba99 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.863505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ebdf-ae3f-4139-8caf-e41e224fba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.899601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5490ebdf-ae3f-4139-8caf-e41e224fba99 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:47.901604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5490ebdf-ae3f-4139-8caf-e41e224fba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:47.955912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 51/103] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:47.957338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc12782-6408-409f-b0a1-8d46d03162c2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:47.957635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc12782-6408-409f-b0a1-8d46d03162c2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:47.957988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc12782-6408-409f-b0a1-8d46d03162c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:47.992913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6bc12782-6408-409f-b0a1-8d46d03162c2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:47.994168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6bc12782-6408-409f-b0a1-8d46d03162c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:48.055587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 53/104] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:48.128350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 52/105] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:48.221306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490ebdf-ae3f-4139-8caf-e41e224fba99 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:48.374083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc12782-6408-409f-b0a1-8d46d03162c2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:48.459650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:48.470581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490ebdf-ae3f-4139-8caf-e41e224fba99 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:48.534228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:48.542764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:48.544232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:48.545036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:48.545834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:48.566576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:48.577985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:48.579706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:48.581255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:48.585993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 54/106] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:48.587902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:48.685185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc12782-6408-409f-b0a1-8d46d03162c2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:48.728645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:48.729903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:48.730191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:48.730466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:48.731596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:48.733513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:48.737710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 53/107] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:48.737710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:48.776406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa5678-f77a-4f7f-8619-64286daf3cd9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:48.818222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000b89f-881a-4f76-8591-3511279dbcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:48.818784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000b89f-881a-4f76-8591-3511279dbcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:48.819192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000b89f-881a-4f76-8591-3511279dbcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:48.819476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000b89f-881a-4f76-8591-3511279dbcb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:48.830807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000b89f-881a-4f76-8591-3511279dbcb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2000b89f-881a-4f76-8591-3511279dbcb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:48.835671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:48.836538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:48.837304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 55/108] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:48.838604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded897c5-cccd-45ed-9477-ad2f3e20c9f0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:48.838604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded897c5-cccd-45ed-9477-ad2f3e20c9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:48.838722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded897c5-cccd-45ed-9477-ad2f3e20c9f0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:48.840613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 56/109] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:48.841176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded897c5-cccd-45ed-9477-ad2f3e20c9f0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:48.859562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:48.870290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a988c4-d0cc-4222-b3cb-d1f36ab291e6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:48.896791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:48.910958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:48.911587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:48.911867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:48.912138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:48.912807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:48.913113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:48.914521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:48.927875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 54/110] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:48.927875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:48.976795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded897c5-cccd-45ed-9477-ad2f3e20c9f0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.022157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.022813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.023095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.023367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.037250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:49.042294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:49.043276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 57/111] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:50:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:49.047203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.062368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316c65d-51d9-4435-b1a1-c3ad802e46ee None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.101867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790c4ff-5e35-42c2-8725-e5e65cca6e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.102431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790c4ff-5e35-42c2-8725-e5e65cca6e31 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.102711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790c4ff-5e35-42c2-8725-e5e65cca6e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.102981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790c4ff-5e35-42c2-8725-e5e65cca6e31 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.103650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790c4ff-5e35-42c2-8725-e5e65cca6e31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:49.105121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790c4ff-5e35-42c2-8725-e5e65cca6e31 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:49.136057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 55/112] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:50:48 2022] POST /identity/v3/projects => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:50:49.137454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133772-4fbe-4279-88c3-8474922f824e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.137752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133772-4fbe-4279-88c3-8474922f824e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.138047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133772-4fbe-4279-88c3-8474922f824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.163287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:49.170048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85c8b5-b7dd-41a2-b122-f81cb45be815 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.171843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-27133772-4fbe-4279-88c3-8474922f824e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:49.174718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-27133772-4fbe-4279-88c3-8474922f824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:49.183806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:49.218610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.219244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.219546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.219825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.220517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:49.220812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:49.222331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:49.235310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 58/113] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:49.236227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.268869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 56/114] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:49.270107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133772-4fbe-4279-88c3-8474922f824e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.400441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c07d7d-a46d-47d7-941a-7135bf314b00 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.448818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfc21e-61b1-43bc-a8fc-e0705bd47fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.450103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfc21e-61b1-43bc-a8fc-e0705bd47fac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.450566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfc21e-61b1-43bc-a8fc-e0705bd47fac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.451014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfc21e-61b1-43bc-a8fc-e0705bd47fac None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.452616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfc21e-61b1-43bc-a8fc-e0705bd47fac None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:49.452750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133772-4fbe-4279-88c3-8474922f824e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.454763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfc21e-61b1-43bc-a8fc-e0705bd47fac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:49.457847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 59/115] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:49.459344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530598-152a-486d-8fad-e2e4576db894 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.459814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530598-152a-486d-8fad-e2e4576db894 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.460266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530598-152a-486d-8fad-e2e4576db894 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.461787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 60/116] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:49.463009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530598-152a-486d-8fad-e2e4576db894 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.508669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.509417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.510225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.510584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.511597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:49.511968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:49.513456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:49.517645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:49.518012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:49.528971 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 57/117] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:49.530546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.614948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530598-152a-486d-8fad-e2e4576db894 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.652161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98c3b8-a0c2-4757-9a71-919421f600ac None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.659651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aae06-1e02-4458-b2f6-b310771a87f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.660502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aae06-1e02-4458-b2f6-b310771a87f8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.660607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aae06-1e02-4458-b2f6-b310771a87f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.660868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aae06-1e02-4458-b2f6-b310771a87f8 None admin] PATH_INFO: `/v3/users/c2509ad4193b4e9ea6083c6302b071f1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.677485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aae06-1e02-4458-b2f6-b310771a87f8 None admin] RBAC: Authorizing `identity:get_user(user_id=c2509ad4193b4e9ea6083c6302b071f1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:49.679050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aae06-1e02-4458-b2f6-b310771a87f8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:49.685329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 61/118] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/users/c2509ad4193b4e9ea6083c6302b071f1 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:49.685329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28f5c3-76fa-4aca-ae63-30c3f03bb1c4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.685329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28f5c3-76fa-4aca-ae63-30c3f03bb1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.685329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28f5c3-76fa-4aca-ae63-30c3f03bb1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.697411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.698221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.698449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.698653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.699851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:49.701325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:49.703176 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 58/119] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:49.703751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1e28f5c3-76fa-4aca-ae63-30c3f03bb1c4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:49.704063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.705479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1e28f5c3-76fa-4aca-ae63-30c3f03bb1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:49.788961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:49.797229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 62/120] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:49.803298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28f5c3-76fa-4aca-ae63-30c3f03bb1c4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.832804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60d2e8-899c-45a9-9266-b5beabc27ded None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.838805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:49.880070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.881012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.881488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.881919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] PATH_INFO: `/v3/users/ca1f33becd334f89a27a7404e04ed5aa` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:49.883946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1f33becd334f89a27a7404e04ed5aa)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:49.885575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:49.892015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 59/121] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/users/ca1f33becd334f89a27a7404e04ed5aa => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:49.893322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:49.943499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28f5c3-76fa-4aca-ae63-30c3f03bb1c4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:49.992047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:49.993525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:49.994670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:49.995416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.010446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:50.017263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897104-aaba-49c9-9f71-53926841c8e6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:50.017459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:50.018369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:50:50.019160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:50.022115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 63/122] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:50.022944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.073778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.076023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.076023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.076223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.081329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:50.153349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:50.170552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1211f-18bf-4c88-b13c-84717725e159 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.174688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:50.176203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:50.217325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.218108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.218581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.219062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/08b02c5702264b2d9c30d73839dd83ba/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.223151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] RBAC: Authorizing `identity:create_grant(project_id=875a7b70c3d04438bc96aa0511c3c007, user_id=08b02c5702264b2d9c30d73839dd83ba, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:50.225542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:50.245559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 64/123] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:50:50 2022] PUT /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/08b02c5702264b2d9c30d73839dd83ba/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:50:50.246858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.321923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 60/124] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:50:50.323455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.378000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:50.410799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a0550-6d38-46c7-bd55-ed1edf548e2c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.459993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.461064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.461561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.461949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.463998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:50.466239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:50.467581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 65/125] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:50 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:50.468336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.494425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b9c86-401b-43d8-838e-b695bb38f21e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.495412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:50.561274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.562221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.562688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.563361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.564919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:50.565400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:50.567372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:50.579866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 61/126] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:50:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:50.581370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.605425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08e7a-ee8d-416e-af14-f1b95052d8c8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.651208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.651976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.652329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.652925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.673838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:50.679745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:50.680938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:50.682079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 66/127] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:50.682847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.727677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:50.736620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b3a30-f256-4fdf-ada8-a31c1c8b9868 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.769696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:50.781709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.782796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.783334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.783811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.785175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:50.787039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:50.789512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 62/128] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:50 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:50.790798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.805382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53a38e-16cf-4f14-b057-646c3997b7e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.845566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.846325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.846613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.846914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/ca1f33becd334f89a27a7404e04ed5aa/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.849399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=875a7b70c3d04438bc96aa0511c3c007, user_id=ca1f33becd334f89a27a7404e04ed5aa, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:50.850993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:50.869022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 67/129] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:50:50 2022] PUT /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/ca1f33becd334f89a27a7404e04ed5aa/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:50:50.870889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.932078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f70507-e6e5-4847-b254-cf1979659e2e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:50.975779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:50.976914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:50.977529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:50.978103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:50.979794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:50.982171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:50.985052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 63/130] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:50 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:50.986490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:50.997891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbca17-dfdc-4065-9f3d-391674ffa4c4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:51.031548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:51.034897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:51.040661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:51.042088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:51.042932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:51.043120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:51.043830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:51.044542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:51.047244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:51.059032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 68/131] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:51.060077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:51.152164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fc626-036d-47ce-8cab-0fecad7022d5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:51.189590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e693899-4b01-470b-bd74-65be8e2fea25 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:51.217096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:51.218803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:51.221036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:51.221569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:51.222569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] RBAC: Adding query filter params (role.id=b206ec3122d24117a73d7addacf3d732, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=c2509ad4193b4e9ea6083c6302b071f1) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:51.223070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:51.225529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:51.233076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 64/132] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:50:50 2022] GET /identity/v3/role_assignments?user.id=c2509ad4193b4e9ea6083c6302b071f1&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=b206ec3122d24117a73d7addacf3d732 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:51.234299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:51.254137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:51.254471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:51.255080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:51.255453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:51.257759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:51.259693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:51.262224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 69/133] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:51 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:51.271055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:51.418082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:51.432934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d0059-5b25-441d-b947-1f05d43736c4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:51.436827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:51.535184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:51.535974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:51.536363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:51.536746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:51.539652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:51.541510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:51.544087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 65/134] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:51 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:51.552300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc66864-5bad-48fa-9e2e-c0d0ec77cc74 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:51.556536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:51.628662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:51.629366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:51.629644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:51.630173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/08b02c5702264b2d9c30d73839dd83ba/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:51.636307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] RBAC: Authorizing `identity:create_grant(project_id=875a7b70c3d04438bc96aa0511c3c007, user_id=08b02c5702264b2d9c30d73839dd83ba, role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:51.640298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:51.674441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 70/135] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:50:51 2022] PUT /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/08b02c5702264b2d9c30d73839dd83ba/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:50:51.757647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3799a8-447b-4dc1-9fea-04b11ab648d9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:51.808267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417678e9-08f5-4f65-9878-3a0d84783e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:51.808926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417678e9-08f5-4f65-9878-3a0d84783e5f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:51.809214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417678e9-08f5-4f65-9878-3a0d84783e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:51.809491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417678e9-08f5-4f65-9878-3a0d84783e5f None admin] PATH_INFO: `/v3/projects/5dc2de4368bd4ce092cfff9965cd7979` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:51.811020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417678e9-08f5-4f65-9878-3a0d84783e5f None admin] RBAC: Authorizing `identity:get_project(project_id=5dc2de4368bd4ce092cfff9965cd7979)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:51.812593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417678e9-08f5-4f65-9878-3a0d84783e5f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:51.814508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 66/136] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:51 2022] GET /identity/v3/projects/5dc2de4368bd4ce092cfff9965cd7979 => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:50:51.820575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:51.984960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e04e8-2d9d-4224-996f-b95466f8487b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:52.013861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c7596-57a2-4753-bede-a0485cc6a450 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:52.014224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c7596-57a2-4753-bede-a0485cc6a450 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:52.014416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c7596-57a2-4753-bede-a0485cc6a450 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:52.016020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 67/137] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:50:52.027457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33cb9-ad75-4970-b337-3351c40b4c72 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:52.027819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33cb9-ad75-4970-b337-3351c40b4c72 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:52.028019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33cb9-ad75-4970-b337-3351c40b4c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:52.032989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:52.072399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a4f33cb9-ad75-4970-b337-3351c40b4c72 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:52.073644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4f33cb9-ad75-4970-b337-3351c40b4c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:52.074548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:52.075492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:52.075977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:52.076447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] PATH_INFO: `/v3/projects/5dc2de4368bd4ce092cfff9965cd7979/users/08b02c5702264b2d9c30d73839dd83ba/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:52.082638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dc2de4368bd4ce092cfff9965cd7979, user_id=08b02c5702264b2d9c30d73839dd83ba, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:52.084509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:52.104036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 71/138] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:50:51 2022] PUT /identity/v3/projects/5dc2de4368bd4ce092cfff9965cd7979/users/08b02c5702264b2d9c30d73839dd83ba/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:50:52.124931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:52.182956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 68/139] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:52.645260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:52.799421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba00b6-e989-4b7c-9970-41a5a3ad6222 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:52.850775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd97fc6-d929-4832-83ed-515be6f4d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:52.852268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd97fc6-d929-4832-83ed-515be6f4d48e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:52.852802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd97fc6-d929-4832-83ed-515be6f4d48e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:52.853270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd97fc6-d929-4832-83ed-515be6f4d48e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:52.858498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd97fc6-d929-4832-83ed-515be6f4d48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:52.860484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd97fc6-d929-4832-83ed-515be6f4d48e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:52.862971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 72/140] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:50:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:52.868856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33cb9-ad75-4970-b337-3351c40b4c72 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:53.004645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33cb9-ad75-4970-b337-3351c40b4c72 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:53.055753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85099a5c-27ad-4a46-afba-c8fceab2e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:53.056774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85099a5c-27ad-4a46-afba-c8fceab2e9b6 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:53.057264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85099a5c-27ad-4a46-afba-c8fceab2e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:53.057752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85099a5c-27ad-4a46-afba-c8fceab2e9b6 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:53.058634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85099a5c-27ad-4a46-afba-c8fceab2e9b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:53.060294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85099a5c-27ad-4a46-afba-c8fceab2e9b6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:53.096471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 69/141] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:50:52 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:50:53.806751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43dfd-9ba5-41fc-ae3a-36b4e92a941f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:53.807443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43dfd-9ba5-41fc-ae3a-36b4e92a941f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:53.807921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43dfd-9ba5-41fc-ae3a-36b4e92a941f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:53.809534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 73/142] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:53.815687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401abc0-cf54-4639-af1b-8523251bb596 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:53.816298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401abc0-cf54-4639-af1b-8523251bb596 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:53.816769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401abc0-cf54-4639-af1b-8523251bb596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:53.839683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3401abc0-cf54-4639-af1b-8523251bb596 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:53.841371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3401abc0-cf54-4639-af1b-8523251bb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:53.863996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:53.934180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 70/143] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:54.346810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f43dfd-9ba5-41fc-ae3a-36b4e92a941f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:54.428649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:54.491799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f43dfd-9ba5-41fc-ae3a-36b4e92a941f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:54.534981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5c44e-b90b-4669-88bf-16d4574d4fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:54.535639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c44e-b90b-4669-88bf-16d4574d4fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:54.535919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c44e-b90b-4669-88bf-16d4574d4fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:54.536190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5c44e-b90b-4669-88bf-16d4574d4fb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:54.548390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5c44e-b90b-4669-88bf-16d4574d4fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-07f5c44e-b90b-4669-88bf-16d4574d4fb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:54.557766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:54.558694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:54.559701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 74/144] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:50:54.568542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401abc0-cf54-4639-af1b-8523251bb596 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:54.577726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8ddc-0b47-480a-a220-58f3793b8ba3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:54.578060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8ddc-0b47-480a-a220-58f3793b8ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:54.578279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b8ddc-0b47-480a-a220-58f3793b8ba3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:54.579753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 75/145] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:54.587126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110bbb2-d51d-4fa3-96c7-46b5842004bc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:54.587443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110bbb2-d51d-4fa3-96c7-46b5842004bc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:54.587680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110bbb2-d51d-4fa3-96c7-46b5842004bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:54.609283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c110bbb2-d51d-4fa3-96c7-46b5842004bc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:54.610531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c110bbb2-d51d-4fa3-96c7-46b5842004bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:54.702940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 76/146] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:54.788736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401abc0-cf54-4639-af1b-8523251bb596 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:54.840215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:54.840869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:54.841152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:54.841423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:54.842148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:54.842453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:54.843910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:54.854806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 71/147] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:54.860843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110bbb2-d51d-4fa3-96c7-46b5842004bc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:54.930464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:55.010231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110bbb2-d51d-4fa3-96c7-46b5842004bc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:55.054314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d246f-ff22-4d61-b7a3-201abdd56cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:55.055044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d246f-ff22-4d61-b7a3-201abdd56cac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:55.055392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d246f-ff22-4d61-b7a3-201abdd56cac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:55.055736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d246f-ff22-4d61-b7a3-201abdd56cac None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:55.057196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d246f-ff22-4d61-b7a3-201abdd56cac None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:55.058884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d246f-ff22-4d61-b7a3-201abdd56cac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:55.060608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 77/148] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:54 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:55.066523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:55.082112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:55.102964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18712097-d39d-4a9d-aad2-03d00ca73e91 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:55.103512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18712097-d39d-4a9d-aad2-03d00ca73e91 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:55.103931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18712097-d39d-4a9d-aad2-03d00ca73e91 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:55.105561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 78/149] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:50:55.113004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023236bc-d1d0-439c-bf4b-e9ff1ff9fb2f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:55.113522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023236bc-d1d0-439c-bf4b-e9ff1ff9fb2f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:55.113992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023236bc-d1d0-439c-bf4b-e9ff1ff9fb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:55.143277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-023236bc-d1d0-439c-bf4b-e9ff1ff9fb2f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:55.144789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-023236bc-d1d0-439c-bf4b-e9ff1ff9fb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:55.246539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 79/150] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:50:55.262500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2abdd-2be6-44fe-b8a1-8bfd6d4833db None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:55.316861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:55.317802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:55.318240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:55.318661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:55.451680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:55.453813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:55.474165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:55.542494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023236bc-d1d0-439c-bf4b-e9ff1ff9fb2f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:55.625525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:55.646295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 72/151] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:50:55.655549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:55.758072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023236bc-d1d0-439c-bf4b-e9ff1ff9fb2f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:55.818465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544c6e8-bdba-4f22-86d9-076fb9b538f5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:55.838046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:55.854906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:55.856135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:55.856735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:55.857325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:55.879979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:55.881200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:55.881637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:55.882078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:55.884078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:55.885806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:55.887736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 73/152] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:55 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:55.888988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:55.889816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:55.904815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:55.905904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:55.906695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 80/153] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:50:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:50:55.916955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:56.027053 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847ea8f-2d4a-4278-b561-0cf3da008e06 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:56.071249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:56.072687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.073140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.073804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.094112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:56.103964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:56.105358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:50:56.106391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:56.109466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 74/154] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:56.111333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:56.214055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:56.253810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:56.269024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150b94e-befe-4caf-841a-731da6c41d3c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:56.357776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de9cb5-d93a-4fa1-aa3e-8ec13c9fd794 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:56.400184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:56.401051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.401468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.401910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.402745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:56.403182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:56.404734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:56.415457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 75/155] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:56.417407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:56.419810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaa5de-0cee-42df-ad88-81a678202007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:56.420809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaa5de-0cee-42df-ad88-81a678202007 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.421288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaa5de-0cee-42df-ad88-81a678202007 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.421776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaa5de-0cee-42df-ad88-81a678202007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.423537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaa5de-0cee-42df-ad88-81a678202007 None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=08b02c5702264b2d9c30d73839dd83ba) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:56.424984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaa5de-0cee-42df-ad88-81a678202007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:56.426778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaa5de-0cee-42df-ad88-81a678202007 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:56.443611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 81/156] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:50:55 2022] GET /identity/v3/role_assignments?user.id=08b02c5702264b2d9c30d73839dd83ba&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 608 bytes in 528 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:56.445696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69678e2-2e42-4ee6-b0ba-ed5f8dcd5123 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.446033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69678e2-2e42-4ee6-b0ba-ed5f8dcd5123 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.446497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69678e2-2e42-4ee6-b0ba-ed5f8dcd5123 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.448872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 82/157] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:56.458200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6d63d-e717-4c7a-9340-c472455a0a07 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.458601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6d63d-e717-4c7a-9340-c472455a0a07 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.458859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6d63d-e717-4c7a-9340-c472455a0a07 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.460567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 83/158] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:56.461129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6d63d-e717-4c7a-9340-c472455a0a07 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:56.562092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b91c-1a66-451b-ba6c-032a96fab073 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:56.566301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:56.624230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a1ccd-50e2-46b0-8d7e-1d389617e550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:56.628132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a1ccd-50e2-46b0-8d7e-1d389617e550 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.628867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a1ccd-50e2-46b0-8d7e-1d389617e550 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.629582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a1ccd-50e2-46b0-8d7e-1d389617e550 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.631293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a1ccd-50e2-46b0-8d7e-1d389617e550 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:56.633284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a1ccd-50e2-46b0-8d7e-1d389617e550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:56.635224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a1ccd-50e2-46b0-8d7e-1d389617e550 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:56.654930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 76/159] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:56.656734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfac0e-3bb7-4e16-b4e1-8a4a404676b3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.663366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfac0e-3bb7-4e16-b4e1-8a4a404676b3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.664860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfac0e-3bb7-4e16-b4e1-8a4a404676b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.705832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fedfac0e-3bb7-4e16-b4e1-8a4a404676b3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:56.707977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fedfac0e-3bb7-4e16-b4e1-8a4a404676b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:56.780742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6d63d-e717-4c7a-9340-c472455a0a07 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:56.832631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a813c3-2496-483c-ba46-f4450c9e0857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:56.833182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a813c3-2496-483c-ba46-f4450c9e0857 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.833459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a813c3-2496-483c-ba46-f4450c9e0857 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.833759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a813c3-2496-483c-ba46-f4450c9e0857 None admin] PATH_INFO: `/v3/users/ca1f33becd334f89a27a7404e04ed5aa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.834948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a813c3-2496-483c-ba46-f4450c9e0857 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1f33becd334f89a27a7404e04ed5aa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:56.836456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a813c3-2496-483c-ba46-f4450c9e0857 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:56.838695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 84/160] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:56 2022] GET /identity/v3/users/ca1f33becd334f89a27a7404e04ed5aa => generated 247 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:56.840179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fd749-1e2d-4b75-9096-e194e83b907b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:56.840504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fd749-1e2d-4b75-9096-e194e83b907b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:56.840800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fd749-1e2d-4b75-9096-e194e83b907b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:56.861629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ea0fd749-1e2d-4b75-9096-e194e83b907b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:56.862870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ea0fd749-1e2d-4b75-9096-e194e83b907b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:56.897034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:56.964147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 85/161] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:56.964956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 77/162] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:56.965742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fd749-1e2d-4b75-9096-e194e83b907b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:56.965962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfac0e-3bb7-4e16-b4e1-8a4a404676b3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:57.126265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:57.139906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfac0e-3bb7-4e16-b4e1-8a4a404676b3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:57.169040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fd749-1e2d-4b75-9096-e194e83b907b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:57.222710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:57.226825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:57.227212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:57.227508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:57.227845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:57.228550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:57.228823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:57.229041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] PATH_INFO: `/v3/users/c2509ad4193b4e9ea6083c6302b071f1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:57.234462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] RBAC: Authorizing `identity:get_user(user_id=c2509ad4193b4e9ea6083c6302b071f1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:57.236017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:57.237739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 86/163] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:56 2022] GET /identity/v3/users/c2509ad4193b4e9ea6083c6302b071f1 => generated 289 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:57.243351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:57.294272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:57.374164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:57.375748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:57.406967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ac882-9232-49c9-991f-c4664c92fefa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:57.454818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:57.455490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:57.455769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:57.456030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:57.457297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:57.458790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:57.460860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 87/164] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:57 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:57.467631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:57.554498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 78/165] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:50:57.556401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:57.566473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:57.641189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:57.653828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a7b5-9241-40ee-881d-f41d469bada0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:57.727295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:57.727981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:57.728256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:57.728523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/c2509ad4193b4e9ea6083c6302b071f1/roles/b206ec3122d24117a73d7addacf3d732` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:57.746363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=875a7b70c3d04438bc96aa0511c3c007, user_id=c2509ad4193b4e9ea6083c6302b071f1, role_id=b206ec3122d24117a73d7addacf3d732)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:57.747867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:57.767917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 88/166] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:50:57 2022] PUT /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007/users/c2509ad4193b4e9ea6083c6302b071f1/roles/b206ec3122d24117a73d7addacf3d732 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:50:57.795152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31870178-827b-4162-a07e-6a491cac23bc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:57.881216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:57.881939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:57.882223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:57.882502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:57.884083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:57.885611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:57.888887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 79/167] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:57 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:57.896397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:57.899614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:58.078464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:58.129656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa473-8f50-4a94-b9a5-6afd95d89bf1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:58.151950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa304b-a7a6-44a7-a5c4-45f57c35b01e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:58.198760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:58.199479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:58.199759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:58.200731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:58.201454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=ca1f33becd334f89a27a7404e04ed5aa) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:58.201800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:58.203287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:58.208839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:58.209799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:58.210192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:58.210599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:58.223816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 89/168] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:50:57 2022] GET /identity/v3/role_assignments?user.id=ca1f33becd334f89a27a7404e04ed5aa&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:58.225994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:58.241268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:58.248786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:58.250528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:50:58.252160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 80/169] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:50:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:58.265538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:58.297433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:58.500553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcee631-670e-4af7-877d-c3e2ba158dba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:58.566032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af398e2-3d69-403f-95ea-ee0a2d155a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:58.567440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af398e2-3d69-403f-95ea-ee0a2d155a56 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:58.569929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af398e2-3d69-403f-95ea-ee0a2d155a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:58.570772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af398e2-3d69-403f-95ea-ee0a2d155a56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:58.589776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714ae9b-8941-4bcb-acd4-bfec45bb4aa2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:58.605320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af398e2-3d69-403f-95ea-ee0a2d155a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4af398e2-3d69-403f-95ea-ee0a2d155a56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:50:58.615622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:50:58.617180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:50:58.618797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:50:58.623793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 90/170] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:50:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:50:58.626713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5f075-4c12-453d-a1ee-102357490a97 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:58.627576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5f075-4c12-453d-a1ee-102357490a97 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:58.628293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5f075-4c12-453d-a1ee-102357490a97 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:58.631095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 91/171] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:58.635086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5f075-4c12-453d-a1ee-102357490a97 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:58.649771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:58.680960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d0e8-9e2a-4372-ab4c-e451bd50c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:58.681622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d0e8-9e2a-4372-ab4c-e451bd50c581 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:58.681940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d0e8-9e2a-4372-ab4c-e451bd50c581 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:58.682296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d0e8-9e2a-4372-ab4c-e451bd50c581 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:58.682903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437d0e8-9e2a-4372-ab4c-e451bd50c581 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:58.683207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437d0e8-9e2a-4372-ab4c-e451bd50c581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:58.684791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437d0e8-9e2a-4372-ab4c-e451bd50c581 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:58.699767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 81/172] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:50:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:58.703206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035f79-de4b-4e84-9857-50fd7d503235 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:58.703587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035f79-de4b-4e84-9857-50fd7d503235 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:58.703871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035f79-de4b-4e84-9857-50fd7d503235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:58.736925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-70035f79-de4b-4e84-9857-50fd7d503235 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:50:58.738451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-70035f79-de4b-4e84-9857-50fd7d503235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:50:58.824213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:58.864384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5f075-4c12-453d-a1ee-102357490a97 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:58.895069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 82/173] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:50:58.895981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035f79-de4b-4e84-9857-50fd7d503235 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:58.912662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:58.913330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:58.913570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:58.913976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:58.914666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:50:58.915023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:58.917044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:58.934031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 92/174] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:50:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:58.935500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:59.116699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035f79-de4b-4e84-9857-50fd7d503235 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:59.134168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:59.156479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fc20a-da42-45fe-9e25-ced065c285aa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:59.176968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:59.177713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:59.177936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:59.178318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:59.182378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:59.183883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:59.185599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 83/175] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:58 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:59.192932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:59.246176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:59.246884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:59.247246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:59.247434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] PATH_INFO: `/v3/users/08b02c5702264b2d9c30d73839dd83ba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:59.254641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] RBAC: Authorizing `identity:get_user(user_id=08b02c5702264b2d9c30d73839dd83ba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:59.256226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:59.258101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 93/176] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:50:58 2022] GET /identity/v3/users/08b02c5702264b2d9c30d73839dd83ba => generated 275 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:59.279429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:59.305852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:59.409226 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ae16-467c-4de6-88a4-588222bf3645 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:59.458961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52332b25-d505-4e5d-94ef-d0a005a4721f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:59.491257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:59.492387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:59.492863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:59.493326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:59.539633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1b6c6-2185-44e9-bf83-3624fa6391ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:59.540721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1b6c6-2185-44e9-bf83-3624fa6391ae None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:59.541218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1b6c6-2185-44e9-bf83-3624fa6391ae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:59.541739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1b6c6-2185-44e9-bf83-3624fa6391ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:59.589939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:59.637943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:59.639828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:59.685345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1b6c6-2185-44e9-bf83-3624fa6391ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:59.688232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1b6c6-2185-44e9-bf83-3624fa6391ae None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:59.726075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:59.793412 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 84/177] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:50:59.799797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:59.805800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 94/178] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:50:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:50:59.807737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0b2bc-1700-4866-9f0c-d39c163a232d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:59.808030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0b2bc-1700-4866-9f0c-d39c163a232d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:59.808310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0b2bc-1700-4866-9f0c-d39c163a232d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:59.810211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 95/179] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:50:59.811025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0b2bc-1700-4866-9f0c-d39c163a232d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:59.915632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:59.938384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0b2bc-1700-4866-9f0c-d39c163a232d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:59.944734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187acf9-fe70-4c2f-a875-ac318b04804a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:50:59.953202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:50:59.981478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:59.982160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:59.982438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:59.982721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] PATH_INFO: `/v3/projects/5dc2de4368bd4ce092cfff9965cd7979` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:50:59.983858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] RBAC: Authorizing `identity:get_project(project_id=5dc2de4368bd4ce092cfff9965cd7979)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:50:59.985340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:50:59.992246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 96/180] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:59 2022] GET /identity/v3/projects/5dc2de4368bd4ce092cfff9965cd7979 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:50:59.992246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:50:59.994614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5de25-5b27-4058-ae6d-516bdba924f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:50:59.995327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5de25-5b27-4058-ae6d-516bdba924f8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:50:59.995607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5de25-5b27-4058-ae6d-516bdba924f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:50:59.995925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5de25-5b27-4058-ae6d-516bdba924f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.008828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5de25-5b27-4058-ae6d-516bdba924f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ead5de25-5b27-4058-ae6d-516bdba924f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:00.013725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:00.014723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 85/181] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:50:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:00.020575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56558949-b9bd-4f1e-b57f-609adf6eddbc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.021358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56558949-b9bd-4f1e-b57f-609adf6eddbc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.021358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56558949-b9bd-4f1e-b57f-609adf6eddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.038454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-56558949-b9bd-4f1e-b57f-609adf6eddbc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:00.039720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-56558949-b9bd-4f1e-b57f-609adf6eddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:00.112451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 86/182] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:00.113881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b33ea-a479-4969-a2a5-2f7657878745 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.114171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b33ea-a479-4969-a2a5-2f7657878745 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.114447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b33ea-a479-4969-a2a5-2f7657878745 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.115737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 87/183] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:00.116501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b33ea-a479-4969-a2a5-2f7657878745 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:00.134522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f551f-9ec3-4bf9-bbd0-2d4ef0e46344 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:00.196230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:00.197249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.197879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.198702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.201503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:00.204178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:00.206192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 97/184] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:50:59 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:00.207179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:00.213269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:00.282860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:00.283901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b33ea-a479-4969-a2a5-2f7657878745 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:00.357945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62f31c-881b-424b-b4c7-b0e58984f7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:00.359313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62f31c-881b-424b-b4c7-b0e58984f7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.359759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548869c1-8909-48ff-8098-33d0a6011bc3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:00.360264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62f31c-881b-424b-b4c7-b0e58984f7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.361652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62f31c-881b-424b-b4c7-b0e58984f7f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.363474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62f31c-881b-424b-b4c7-b0e58984f7f1 None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=5dc2de4368bd4ce092cfff9965cd7979, user.id=08b02c5702264b2d9c30d73839dd83ba) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:00.364250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62f31c-881b-424b-b4c7-b0e58984f7f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:00.367562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62f31c-881b-424b-b4c7-b0e58984f7f1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:00.399529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 88/185] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:00 2022] GET /identity/v3/role_assignments?user.id=08b02c5702264b2d9c30d73839dd83ba&scope.project.id=5dc2de4368bd4ce092cfff9965cd7979&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:00.399810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:00.400643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538868b-536e-45d3-87c0-8351811adcac None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.400819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.401081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538868b-536e-45d3-87c0-8351811adcac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.401229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.401540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538868b-536e-45d3-87c0-8351811adcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.401730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.402704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:00.403164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:00.413712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:00.418791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 98/186] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:00.420433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:00.422026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0538868b-536e-45d3-87c0-8351811adcac None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:00.423234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0538868b-536e-45d3-87c0-8351811adcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:00.506967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 89/187] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:00.507776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538868b-536e-45d3-87c0-8351811adcac None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:00.599155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86ff54-4520-4065-936d-3f1d5fa72938 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:00.618938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:00.622390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:00.634872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538868b-536e-45d3-87c0-8351811adcac None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:00.672974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:00.673605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.674290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.674566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.675321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] RBAC: Adding query filter params (role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=08b02c5702264b2d9c30d73839dd83ba) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:00.675634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:00.677207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:00.684270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:00.684939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.685219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.685495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.686700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:00.689207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:00.692599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 99/189] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:00 2022] GET /identity/v3/role_assignments?user.id=08b02c5702264b2d9c30d73839dd83ba&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:00.692800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 90/189] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:00.693739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:00.698950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:00.910777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482e278-b50f-4aa2-b035-2e88b53c99f1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:00.950891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c9ec3-c2cf-4f7c-8a2d-e942a83a9747 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:00.976061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:00.990687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:00.993588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:00.993966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:00.994514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:00.995681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:01.005596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:01.019939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:01.021799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.022335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.023386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] PATH_INFO: `/v3/users/793aa9c0eec048fd960ac064499ab341` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.038197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 91/190] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:00 2022] POST /identity/v3/users => generated 297 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:51:01.039139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:01.043151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] RBAC: Authorizing `identity:get_user(user_id=793aa9c0eec048fd960ac064499ab341)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:01.045126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:01.047266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 100/191] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:00 2022] GET /identity/v3/users/793aa9c0eec048fd960ac064499ab341 => generated 283 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:01.054265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:01.107720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:01.221392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f11b2-ece5-4999-b012-de593b42d89e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:01.223099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d81ae-fe4b-49bd-83a7-8eb5740dabed None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:01.268415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:01.269156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.269438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.269739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.275511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621d85d-195f-4960-8cd4-bb46bc3e7614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:01.276163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d85d-195f-4960-8cd4-bb46bc3e7614 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.276446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d85d-195f-4960-8cd4-bb46bc3e7614 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.276743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621d85d-195f-4960-8cd4-bb46bc3e7614 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.288138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621d85d-195f-4960-8cd4-bb46bc3e7614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3621d85d-195f-4960-8cd4-bb46bc3e7614 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:01.294640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:01.295646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:01.296457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 92/192] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:01.297117 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91b17e-e2f9-447f-9c9b-0d9091b1df1d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.297516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91b17e-e2f9-447f-9c9b-0d9091b1df1d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.301990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91b17e-e2f9-447f-9c9b-0d9091b1df1d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.303480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 93/193] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:01.304288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91b17e-e2f9-447f-9c9b-0d9091b1df1d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:01.381492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:01.383813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:01.384099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:01.444304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:01.461640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91b17e-e2f9-447f-9c9b-0d9091b1df1d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:01.514238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a353b-ccc7-4970-b8c5-fd620729de89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:01.514762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a353b-ccc7-4970-b8c5-fd620729de89 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.515047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a353b-ccc7-4970-b8c5-fd620729de89 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.515325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a353b-ccc7-4970-b8c5-fd620729de89 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.516017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a353b-ccc7-4970-b8c5-fd620729de89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:01.516325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a353b-ccc7-4970-b8c5-fd620729de89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:01.518661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a353b-ccc7-4970-b8c5-fd620729de89 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:01.532238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 94/194] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:01.538516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562670c-7f1d-4f84-ba26-f67d9cf98eca None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.538807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562670c-7f1d-4f84-ba26-f67d9cf98eca None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.539088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562670c-7f1d-4f84-ba26-f67d9cf98eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.555510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 101/195] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:01.556688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:01.561552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6562670c-7f1d-4f84-ba26-f67d9cf98eca None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:01.563148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6562670c-7f1d-4f84-ba26-f67d9cf98eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:01.662014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 95/196] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:01.662959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562670c-7f1d-4f84-ba26-f67d9cf98eca None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:01.675412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d3b8-c3a4-4337-9d4c-4fd5469d2ae3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:01.714091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:01.725598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:01.726336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:01.727701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.728196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.728658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] PATH_INFO: `/v3/users/793aa9c0eec048fd960ac064499ab341` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.730583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] RBAC: Authorizing `identity:get_user(user_id=793aa9c0eec048fd960ac064499ab341)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:01.732841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:01.734887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 102/197] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:01 2022] GET /identity/v3/users/793aa9c0eec048fd960ac064499ab341 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:01.742683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:01.792268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562670c-7f1d-4f84-ba26-f67d9cf98eca None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:01.831575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:01.832242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.832527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.832802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.834082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:01.835937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:01.837613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 96/198] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:01 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:01.850981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:01.876108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839964e-ae2d-49dc-a059-07c78793ae59 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:01.948339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:01.949288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:01.949751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:01.950207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:01.982719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9db53-04e2-42c4-b49d-19fecbeee4f0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:02.029185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:02.032648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:02.033948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:02.034422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:02.034923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:02.036040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=793aa9c0eec048fd960ac064499ab341) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:02.036454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:02.038067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:02.045205 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:02.051301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 97/199] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:01 2022] GET /identity/v3/role_assignments?user.id=793aa9c0eec048fd960ac064499ab341&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:02.051301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:02.063940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:02.068820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:02.281119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2201e29-aa33-4a0b-8d7c-e373aea17efd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:02.308488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 103/200] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:02.318545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:02.330286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:02.331397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:02.331932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:02.332384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:02.351317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:02.359587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:02.360569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:02.361597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:02.363991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 98/201] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:02.370320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:02.410829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:02.503451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0970e1c-a39f-460c-8c42-47783378e809 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:02.552106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b45d4a-dc22-4c6c-bfe9-b3375cbbe0be None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:02.558975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:02.559850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:02.560276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:02.560787 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:02.561411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:02.561878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:02.562391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:02.564206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:02.575504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 99/202] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:02.581791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:02.628398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:02.629397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:02.629756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:02.630179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:02.634703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:02.636270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:02.638444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 104/203] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:02 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:02.646058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:02.735598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:02.746228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c307f-d939-4051-93a0-59e8be091a3c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:02.805466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:02.806536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:02.808071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:02.808665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] PATH_INFO: `/v3/users/c2509ad4193b4e9ea6083c6302b071f1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:02.811567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] RBAC: Authorizing `identity:get_user(user_id=c2509ad4193b4e9ea6083c6302b071f1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:02.813358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:02.816405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 100/204] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:02 2022] GET /identity/v3/users/c2509ad4193b4e9ea6083c6302b071f1 => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:02.826502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:02.883917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd670e-c07f-4775-9e56-c1a5082707b6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:02.965991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:02.966718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:02.967164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:02.967519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:02.968358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] RBAC: Adding query filter params (role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=793aa9c0eec048fd960ac064499ab341) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:02.968768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:02.970458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:02.981862 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 105/205] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:02 2022] GET /identity/v3/role_assignments?user.id=793aa9c0eec048fd960ac064499ab341&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:02.997354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b9ad6-615c-4e91-8be8-3bda52ecc6f0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:03.045209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:03.046067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:03.046360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:03.046639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:03.109867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:03.171171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:03.173088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:03.319303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 101/206] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:03.328666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:03.386499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:03.510545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e04f8-f5fb-43ce-86a2-0bdd96b65ae3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:03.586628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc3bb-45d4-4489-b3d2-4dd5a42cca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:03.587690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc3bb-45d4-4489-b3d2-4dd5a42cca15 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:03.588168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc3bb-45d4-4489-b3d2-4dd5a42cca15 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:03.588639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc3bb-45d4-4489-b3d2-4dd5a42cca15 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:03.590830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc3bb-45d4-4489-b3d2-4dd5a42cca15 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:03.593630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc3bb-45d4-4489-b3d2-4dd5a42cca15 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:03.601241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 106/207] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:03 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:03.606737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:03.708731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:03.756039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7446ba2-0e95-4d45-8281-06fdd871cfa3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:03.795577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024c96e-e9f0-4380-9f98-837dd6f93c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:03.796206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024c96e-e9f0-4380-9f98-837dd6f93c82 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:03.796483 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024c96e-e9f0-4380-9f98-837dd6f93c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:03.796751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024c96e-e9f0-4380-9f98-837dd6f93c82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:03.797479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024c96e-e9f0-4380-9f98-837dd6f93c82 None admin] RBAC: Adding query filter params (role.id=b206ec3122d24117a73d7addacf3d732, scope.project.id=875a7b70c3d04438bc96aa0511c3c007, user.id=c2509ad4193b4e9ea6083c6302b071f1) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:03.797817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024c96e-e9f0-4380-9f98-837dd6f93c82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:03.799357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024c96e-e9f0-4380-9f98-837dd6f93c82 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:03.806211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 102/208] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:03 2022] GET /identity/v3/role_assignments?user.id=c2509ad4193b4e9ea6083c6302b071f1&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=b206ec3122d24117a73d7addacf3d732 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:04.204500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74383ba-27e6-406b-9d02-54ad1c3e9bfe None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:04.204962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74383ba-27e6-406b-9d02-54ad1c3e9bfe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:04.205497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74383ba-27e6-406b-9d02-54ad1c3e9bfe None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:04.208059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 107/209] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:04.215721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259dab5-766e-4f44-a173-a26547b16c02 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:04.216162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259dab5-766e-4f44-a173-a26547b16c02 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:04.216628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259dab5-766e-4f44-a173-a26547b16c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:04.255518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9259dab5-766e-4f44-a173-a26547b16c02 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:04.257823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9259dab5-766e-4f44-a173-a26547b16c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:04.397894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 103/210] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:04.562328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abdcc9-043e-4346-bfa5-c588136bde39 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:04.563931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abdcc9-043e-4346-bfa5-c588136bde39 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:04.564770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abdcc9-043e-4346-bfa5-c588136bde39 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:04.567641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 108/211] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:04.575944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76610777-b7c8-4ee6-8ebf-b20520c58f93 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:04.576812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76610777-b7c8-4ee6-8ebf-b20520c58f93 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:04.577593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76610777-b7c8-4ee6-8ebf-b20520c58f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:04.611795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-76610777-b7c8-4ee6-8ebf-b20520c58f93 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:04.614354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-76610777-b7c8-4ee6-8ebf-b20520c58f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:04.708630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:04.740100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 104/212] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:04.927644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abdcc9-043e-4346-bfa5-c588136bde39 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:05.024880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76610777-b7c8-4ee6-8ebf-b20520c58f93 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:05.117604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:05.124752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abdcc9-043e-4346-bfa5-c588136bde39 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:05.180289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76610777-b7c8-4ee6-8ebf-b20520c58f93 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:05.207708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:05.209220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.210852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.211815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.214497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:05.217163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:05.219216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 109/213] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:05.231060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:05.236991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:05.237726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.238006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.238279 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.252438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:05.257688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:05.259216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:05.261368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 105/214] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:05.266863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:05.350877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:05.399871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af66d46-9beb-4755-a5f0-48b6f0856cf0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:05.441050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e4e96-f437-4885-8be8-eb8513bb1156 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:05.459130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:05.460589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.461152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.462075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.463019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:05.465107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:05.508387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb2c65-73f4-446d-bc82-072765733761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:05.510143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb2c65-73f4-446d-bc82-072765733761 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.510765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb2c65-73f4-446d-bc82-072765733761 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.511468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb2c65-73f4-446d-bc82-072765733761 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.512950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb2c65-73f4-446d-bc82-072765733761 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:05.513783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb2c65-73f4-446d-bc82-072765733761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:05.517165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb2c65-73f4-446d-bc82-072765733761 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:05.518331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 110/215] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:05 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:51:05.534607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 106/216] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:05.543788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:05.587675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:05.640155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb2f23-28e0-409e-a32f-c998847aac5e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.640903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb2f23-28e0-409e-a32f-c998847aac5e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.641469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb2f23-28e0-409e-a32f-c998847aac5e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.643366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 107/217] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:51:05.651839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9ba2-7f5b-43e6-aa49-68e83ed34c77 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.652210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9ba2-7f5b-43e6-aa49-68e83ed34c77 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.652622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9ba2-7f5b-43e6-aa49-68e83ed34c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.682174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ef8d9ba2-7f5b-43e6-aa49-68e83ed34c77 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:05.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ef8d9ba2-7f5b-43e6-aa49-68e83ed34c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:05.716648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64313a1-7088-4737-bbc8-f33cf1527a7f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:05.760341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:05.761105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.761390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.761683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] PATH_INFO: `/v3/users/65d7c478401d484cae6ca361a78ebc59` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.785204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] RBAC: Authorizing `identity:get_user(user_id=65d7c478401d484cae6ca361a78ebc59)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:05.787926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:05.794714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 111/218] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:05 2022] GET /identity/v3/users/65d7c478401d484cae6ca361a78ebc59 => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:05.796862 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:05.821797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 108/219] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:05.821797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96d3be-821a-4eb9-a6bb-7fd607eb633a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.821797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96d3be-821a-4eb9-a6bb-7fd607eb633a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.821797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96d3be-821a-4eb9-a6bb-7fd607eb633a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.822858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 109/220] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:05.833652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a91da-de2f-4a63-aaa5-fc3351663445 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:05.834002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a91da-de2f-4a63-aaa5-fc3351663445 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:05.834369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a91da-de2f-4a63-aaa5-fc3351663445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:05.860599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d21a91da-de2f-4a63-aaa5-fc3351663445 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:05.862436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d21a91da-de2f-4a63-aaa5-fc3351663445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:05.919407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:05.951911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:05.984420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 110/221] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:05.993943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e032e-bd65-4bdc-9cf4-e3744a16ec72 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:06.044123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:06.045108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:06.045561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:06.046145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:06.123657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a91da-de2f-4a63-aaa5-fc3351663445 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:06.171064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:06.174383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:06.290496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:06.328082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 112/222] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:06.329974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:06.394001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a91da-de2f-4a63-aaa5-fc3351663445 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:06.462829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:06.463905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:06.464186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:06.464571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:06.483595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:06.493098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:06.494147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:06.494983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:06.497716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 111/223] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:06.497716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:06.512120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ca1fd-5eab-4425-9525-972ebf9dacbe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:06.542284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:06.593313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:06.594081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:06.594285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:06.594558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:06.611509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:06.619081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:06.620269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 113/224] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:06.621957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:06.647952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef07b4-c4ea-4187-aaaa-fd69db2838ee None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:06.678901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:06.702172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:06.702838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:06.703119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:06.703388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:06.704902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:06.707088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:06.710021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 112/225] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:06 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:06.711180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:06.763764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a94406-e9bb-4d0f-8d78-c970748306ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:06.818013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:06.819339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:06.819902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:06.820549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:06.821569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:06.822240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:06.824609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:06.842504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 114/226] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:06.844844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:06.864644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:06.875426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f128d5-6243-49e8-ad7d-495bdf134e04 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:06.933090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:06.933754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:06.934037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:06.934309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:06.935024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:06.935848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:06.937270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:06.950927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 113/227] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:06.951822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:07.057147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f92683-ad03-4dc4-97f0-c4f3a5145685 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:07.060088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:07.132589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:07.134614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:07.135103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:07.135794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:07.137415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=65d7c478401d484cae6ca361a78ebc59) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:07.138221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:07.142047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:07.144236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884eeed3-0207-450a-aef4-721161664f95 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:07.154997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 115/228] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:06 2022] GET /identity/v3/role_assignments?user.id=65d7c478401d484cae6ca361a78ebc59&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:07.156255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:07.247843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:07.250717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:07.252784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:07.253639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] PATH_INFO: `/v3/users/793aa9c0eec048fd960ac064499ab341` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:07.256152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] RBAC: Authorizing `identity:get_user(user_id=793aa9c0eec048fd960ac064499ab341)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:07.263266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:07.267362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 114/229] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:06 2022] GET /identity/v3/users/793aa9c0eec048fd960ac064499ab341 => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:07.274144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:07.355749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:07.401693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ce0fc-89e1-4775-bf53-22c6de721afe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:07.490231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:07.490580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:07.490840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:07.491116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] PATH_INFO: `/v3/users/793aa9c0eec048fd960ac064499ab341` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:07.494523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] RBAC: Authorizing `identity:get_user(user_id=793aa9c0eec048fd960ac064499ab341)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:07.496198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:07.499583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 116/230] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:07 2022] GET /identity/v3/users/793aa9c0eec048fd960ac064499ab341 => generated 283 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:07.512936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:07.518647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c1c6e-e93f-4415-8ae7-4b66fa6753a8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:07.569027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:07.573779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:07.575317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:07.576550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:07.577084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:07.578702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:07.580970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:07.583264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 115/231] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:07 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:07.589243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:07.666183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6ce4b-b30f-4b3a-b71f-5d839a6f0485 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:07.712142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:07.713031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:07.713312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:07.713577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:07.715142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:07.716643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:07.719785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 117/232] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:07 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:07.730490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:07.780600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:07.797993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195e928-914b-428c-9917-d0ba16427702 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:07.875778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72353a1d-59fc-4923-8826-2805130fb578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:07.877381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72353a1d-59fc-4923-8826-2805130fb578 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:07.878050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72353a1d-59fc-4923-8826-2805130fb578 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:07.878535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72353a1d-59fc-4923-8826-2805130fb578 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522/users/793aa9c0eec048fd960ac064499ab341/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:07.881016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72353a1d-59fc-4923-8826-2805130fb578 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de48b6c426840f4933ff94a73b59522, user_id=793aa9c0eec048fd960ac064499ab341, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:07.882859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72353a1d-59fc-4923-8826-2805130fb578 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:07.904637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 116/233] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:51:07 2022] PUT /identity/v3/projects/4de48b6c426840f4933ff94a73b59522/users/793aa9c0eec048fd960ac064499ab341/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:51:07.941732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36042ae8-70af-4b26-918c-1d9bc0f36832 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:07.944231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:07.998624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c67b9-8fd4-4033-b660-9d005279160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:07.999783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c67b9-8fd4-4033-b660-9d005279160b None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:08.000389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c67b9-8fd4-4033-b660-9d005279160b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:08.000933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c67b9-8fd4-4033-b660-9d005279160b None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522/users/793aa9c0eec048fd960ac064499ab341/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:08.004669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c67b9-8fd4-4033-b660-9d005279160b None admin] RBAC: Authorizing `identity:create_grant(project_id=4de48b6c426840f4933ff94a73b59522, user_id=793aa9c0eec048fd960ac064499ab341, role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:08.006982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c67b9-8fd4-4033-b660-9d005279160b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:08.028455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 118/234] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:51:07 2022] PUT /identity/v3/projects/4de48b6c426840f4933ff94a73b59522/users/793aa9c0eec048fd960ac064499ab341/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:51:08.839952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9cb3c-ab89-4b92-be39-50e93494738b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:08.840765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9cb3c-ab89-4b92-be39-50e93494738b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:08.840765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9cb3c-ab89-4b92-be39-50e93494738b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:08.842597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 117/235] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:08.848835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3b98d-82cf-4cce-98f6-10cc52aa3db0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:08.849367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3b98d-82cf-4cce-98f6-10cc52aa3db0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:08.849367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3b98d-82cf-4cce-98f6-10cc52aa3db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:08.876589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eae3b98d-82cf-4cce-98f6-10cc52aa3db0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:08.878119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eae3b98d-82cf-4cce-98f6-10cc52aa3db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:08.975456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 119/236] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:09.318175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9cb3c-ab89-4b92-be39-50e93494738b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:09.521215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9cb3c-ab89-4b92-be39-50e93494738b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:09.535445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cea0-0255-4f9c-9010-104955e860d2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:09.535939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cea0-0255-4f9c-9010-104955e860d2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:09.536222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cea0-0255-4f9c-9010-104955e860d2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:09.542751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 120/237] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:09.551489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c6dae-618d-46de-8b1a-2002d7bc7216 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:09.551797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c6dae-618d-46de-8b1a-2002d7bc7216 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:09.552043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c6dae-618d-46de-8b1a-2002d7bc7216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:09.592711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6a5c6dae-618d-46de-8b1a-2002d7bc7216 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:09.594396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6a5c6dae-618d-46de-8b1a-2002d7bc7216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:09.628087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:09.629526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:09.630091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:09.630602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:09.663549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:09.686571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:09.688347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:09.690512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:09.695141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 118/238] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:09.707002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:09.725325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 121/239] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:09.762300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:09.958464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ba535-8872-4389-a996-e95a136e1222 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:10.027931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:10.028567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:10.028844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:10.029120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:10.029823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:10.030124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:10.031526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:10.046574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 119/240] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:10.052856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c6dae-618d-46de-8b1a-2002d7bc7216 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:10.115175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:10.226826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c6dae-618d-46de-8b1a-2002d7bc7216 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:10.273732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:10.274406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:10.274686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:10.274957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] PATH_INFO: `/v3/users/29f5cdba84184c89ba98cabf060fc48f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:10.277947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:10.292539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] RBAC: Authorizing `identity:get_user(user_id=29f5cdba84184c89ba98cabf060fc48f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:10.294170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:10.297884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 122/241] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:10 2022] GET /identity/v3/users/29f5cdba84184c89ba98cabf060fc48f => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:10.313725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:10.426543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:10.452579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7480d-bc43-41c0-8fc0-6baa293f7f32 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:10.470599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:10.503177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcb3b0-6dac-4054-a066-431205155efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:10.504666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcb3b0-6dac-4054-a066-431205155efd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:10.505202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcb3b0-6dac-4054-a066-431205155efd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:10.505736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcb3b0-6dac-4054-a066-431205155efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:10.553062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeaa0f2-dfae-4273-9c7f-bc0e5fd4275f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:10.614958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcb3b0-6dac-4054-a066-431205155efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:10.616838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcb3b0-6dac-4054-a066-431205155efd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:10.680950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36595f6-7ac8-443c-b342-524cbd4db616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:10.681961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36595f6-7ac8-443c-b342-524cbd4db616 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:10.682374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36595f6-7ac8-443c-b342-524cbd4db616 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:10.682771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36595f6-7ac8-443c-b342-524cbd4db616 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:10.684586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36595f6-7ac8-443c-b342-524cbd4db616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:10.687082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36595f6-7ac8-443c-b342-524cbd4db616 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:10.690745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 120/242] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:10.692918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788c2b9-9522-46a3-9b70-1672d05819ac None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:10.693383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788c2b9-9522-46a3-9b70-1672d05819ac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:10.701070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788c2b9-9522-46a3-9b70-1672d05819ac None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:10.704200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 121/243] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:10.705568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788c2b9-9522-46a3-9b70-1672d05819ac None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:10.741900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:10.771481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 123/244] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:10.772813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3316840-13a7-42e5-ab9c-c9c5a11548a4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:10.773140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3316840-13a7-42e5-ab9c-c9c5a11548a4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:10.773421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3316840-13a7-42e5-ab9c-c9c5a11548a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:10.796217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e3316840-13a7-42e5-ab9c-c9c5a11548a4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:10.798993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e3316840-13a7-42e5-ab9c-c9c5a11548a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:10.901065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 124/245] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:10.902412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3316840-13a7-42e5-ab9c-c9c5a11548a4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:10.911503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788c2b9-9522-46a3-9b70-1672d05819ac None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:10.983016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:10.984176 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:10.989456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:10.989456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:10.989456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:10.989456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:11.029031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:11.041563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 122/246] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:10 2022] POST /identity/v3/users => generated 293 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:51:11.140768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3316840-13a7-42e5-ab9c-c9c5a11548a4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:11.164289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:11.213824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7064711-6764-4d5c-8570-276fe5235e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:11.214907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7064711-6764-4d5c-8570-276fe5235e9d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:11.215405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7064711-6764-4d5c-8570-276fe5235e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:11.215879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7064711-6764-4d5c-8570-276fe5235e9d None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:11.218195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7064711-6764-4d5c-8570-276fe5235e9d None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:11.221183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7064711-6764-4d5c-8570-276fe5235e9d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:11.224675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 125/247] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:10 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:11.232072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:11.321585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627601e-7920-4f74-8844-3bbf899e5711 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:11.322821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627601e-7920-4f74-8844-3bbf899e5711 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:11.323540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627601e-7920-4f74-8844-3bbf899e5711 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:11.327053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 126/248] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:51:11.339282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9cf08-1039-445a-9aeb-4cb224f581aa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:11.340337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9cf08-1039-445a-9aeb-4cb224f581aa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:11.340817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9cf08-1039-445a-9aeb-4cb224f581aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:11.364339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-35e9cf08-1039-445a-9aeb-4cb224f581aa None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:11.366402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-35e9cf08-1039-445a-9aeb-4cb224f581aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:11.476094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 127/249] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:11.476967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9cf08-1039-445a-9aeb-4cb224f581aa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:11.487647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9449-6b87-4ef1-b5f4-e20020e9a2e9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:11.551750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:11.552379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:11.552658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:11.552932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:11.553784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] RBAC: Adding query filter params (role.id=b206ec3122d24117a73d7addacf3d732, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=29f5cdba84184c89ba98cabf060fc48f) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:11.554090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:11.555724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:11.564764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 123/250] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:11 2022] GET /identity/v3/role_assignments?user.id=29f5cdba84184c89ba98cabf060fc48f&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=b206ec3122d24117a73d7addacf3d732 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:11.677837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9cf08-1039-445a-9aeb-4cb224f581aa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:11.698207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:11.736405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:11.738878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:11.739192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:11.739437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:11.755439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:11.760943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:11.762820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:11.764452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 128/251] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:11.772631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:11.853188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:11.903348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:12.011270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66357c62-82b9-4b8b-87fe-d8f1e45eb752 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:12.058378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fde5c-4e1b-44d2-8579-13ddd2cab6b6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:12.063538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb78949-b760-4f86-b1a5-8e741ed316fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:12.064421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78949-b760-4f86-b1a5-8e741ed316fd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.064913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78949-b760-4f86-b1a5-8e741ed316fd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.065353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78949-b760-4f86-b1a5-8e741ed316fd None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.066221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb78949-b760-4f86-b1a5-8e741ed316fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:12.066672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb78949-b760-4f86-b1a5-8e741ed316fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:12.068217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb78949-b760-4f86-b1a5-8e741ed316fd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:12.078981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 124/252] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:12.081400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c2a45-bbef-45bd-bf5a-41c2ac55ef73 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.082211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c2a45-bbef-45bd-bf5a-41c2ac55ef73 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.082885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c2a45-bbef-45bd-bf5a-41c2ac55ef73 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.085740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 125/253] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:12.094086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451dc7b-37a6-46fb-b8e7-90ddddab316f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.094923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451dc7b-37a6-46fb-b8e7-90ddddab316f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.095673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451dc7b-37a6-46fb-b8e7-90ddddab316f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.126354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3451dc7b-37a6-46fb-b8e7-90ddddab316f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:12.128740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3451dc7b-37a6-46fb-b8e7-90ddddab316f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:12.131752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:12.133165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.133496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.134060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.153304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:12.162832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:12.164850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 129/254] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:51:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:12.166793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:12.252696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:12.268548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 126/255] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:12.270301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451dc7b-37a6-46fb-b8e7-90ddddab316f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:12.376440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:12.466462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451dc7b-37a6-46fb-b8e7-90ddddab316f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:12.486010 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cf182-55d8-41c8-8036-3cf89614cc5a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:12.539314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:12.540387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.540880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.541365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.542691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:12.543318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:12.545035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:12.545993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:12.546322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.546813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.547312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] PATH_INFO: `/v3/users/65d7c478401d484cae6ca361a78ebc59` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.549760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] RBAC: Authorizing `identity:get_user(user_id=65d7c478401d484cae6ca361a78ebc59)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:12.553136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:12.556367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 130/256] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:12 2022] GET /identity/v3/users/65d7c478401d484cae6ca361a78ebc59 => generated 297 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:12.568228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 127/257] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:12.570704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:12.574895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:12.758554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2820c-464c-4655-aa67-b30c287d3c17 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:12.762150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:12.834913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:12.835574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.835849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.836143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.837591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:12.839233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:12.841276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 131/258] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:12 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:12.853097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:12.917422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03ca34-04e7-41f2-9058-b0ab5af541d5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:12.958640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:12.987643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:12.988648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:12.991251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:12.991746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] PATH_INFO: `/v3/users/793aa9c0eec048fd960ac064499ab341` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:12.993202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] RBAC: Authorizing `identity:get_user(user_id=793aa9c0eec048fd960ac064499ab341)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:12.995670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:13.004882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 128/259] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:12 2022] GET /identity/v3/users/793aa9c0eec048fd960ac064499ab341 => generated 283 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:13.009944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:13.100329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d0f2-ff95-4a6a-a329-a80783e98e79 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:13.174708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6c482-ac28-49c4-938d-a0614f972be8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:13.211687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:13.212434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:13.212716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:13.212987 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522/users/65d7c478401d484cae6ca361a78ebc59/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:13.220148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de48b6c426840f4933ff94a73b59522, user_id=65d7c478401d484cae6ca361a78ebc59, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:13.223158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:13.226622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:13.227844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:13.228433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:13.230946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:13.255936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 132/260] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:51:12 2022] PUT /identity/v3/projects/4de48b6c426840f4933ff94a73b59522/users/65d7c478401d484cae6ca361a78ebc59/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:13.262362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:13.340700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:13.471295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:13.542892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:13.545481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:13.576488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473e203-117b-43bd-b304-6fca48e29202 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:13.653314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:13.653990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:13.654349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:13.654675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:13.684902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:13.691458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:13.693183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:13.694771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 133/261] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:13.703459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:13.865877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 129/262] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 857 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:13.870297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:13.904302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:13.927497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9360fd-eef3-4ece-ae1a-a0815f58b9f5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:14.000878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:14.001643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:14.002094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:14.002642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:14.003816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:14.004307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:14.006408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:14.017420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 134/263] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:14.039086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:14.065995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:14.094876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c256871-d878-43fb-aefc-ff4669eee4ab None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:14.159449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:14.160510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:14.161057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:14.161602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:14.164001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:14.166757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:14.170140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 130/264] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:13 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:14.182811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:14.233762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073b133-0a4b-4541-8964-cfc117543f10 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:14.282191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:14.282900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:14.283180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:14.283458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] PATH_INFO: `/v3/users/793aa9c0eec048fd960ac064499ab341` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:14.284679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] RBAC: Authorizing `identity:get_user(user_id=793aa9c0eec048fd960ac064499ab341)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:14.286207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:14.288186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 135/265] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:14 2022] GET /identity/v3/users/793aa9c0eec048fd960ac064499ab341 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:14.296813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:14.322055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:14.443081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e53e3-e8fb-453b-9a14-8cb738c1109b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:14.475798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed89808-1425-4855-9e9f-34bce0b4a912 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:14.544957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:14.545598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:14.545909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:14.546189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:14.579796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:14.580528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:14.580821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:14.581069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:14.582004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=793aa9c0eec048fd960ac064499ab341) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:14.582344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:14.583919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:14.597238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 131/266] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:14 2022] GET /identity/v3/role_assignments?user.id=793aa9c0eec048fd960ac064499ab341&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:14.640388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:14.642078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:14.661037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:14.772786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 136/267] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:14.778931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:14.820619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:14.968251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca11216-8ec2-4976-88c9-e93a3eeb8686 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:15.027579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367f179-d3b0-4771-bbde-742171534c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:15.028409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367f179-d3b0-4771-bbde-742171534c2a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.028816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367f179-d3b0-4771-bbde-742171534c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.029229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367f179-d3b0-4771-bbde-742171534c2a None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.031162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367f179-d3b0-4771-bbde-742171534c2a None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:15.034080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367f179-d3b0-4771-bbde-742171534c2a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:15.036916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 132/268] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:14 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:15.043774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:15.252282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea7d04-32a7-4f1a-84fd-e5c27edd5bbd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:15.286367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:15.317386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fd393-368e-4886-85b8-60079e8357d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:15.318052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd393-368e-4886-85b8-60079e8357d2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.318369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd393-368e-4886-85b8-60079e8357d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.318666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd393-368e-4886-85b8-60079e8357d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.319442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fd393-368e-4886-85b8-60079e8357d2 None admin] RBAC: Adding query filter params (role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=793aa9c0eec048fd960ac064499ab341) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:15.319894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fd393-368e-4886-85b8-60079e8357d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:15.321979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fd393-368e-4886-85b8-60079e8357d2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:15.330032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 137/269] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:15 2022] GET /identity/v3/role_assignments?user.id=793aa9c0eec048fd960ac064499ab341&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:15.366363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6458a-8d94-403e-a231-d8aeb7186371 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.366955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6458a-8d94-403e-a231-d8aeb7186371 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.366955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6458a-8d94-403e-a231-d8aeb7186371 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.368746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 133/270] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:15.381480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71230f1-2c08-4649-99a6-af600c68edf6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.381791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71230f1-2c08-4649-99a6-af600c68edf6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.382361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71230f1-2c08-4649-99a6-af600c68edf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.404000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b71230f1-2c08-4649-99a6-af600c68edf6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:15.405321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b71230f1-2c08-4649-99a6-af600c68edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:15.516366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 138/271] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:15.665919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562fca-4a50-4761-803c-55aa9cc0b1fe None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.666316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562fca-4a50-4761-803c-55aa9cc0b1fe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.667133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562fca-4a50-4761-803c-55aa9cc0b1fe None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.670402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 134/272] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:15.679992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a64b6e-c045-4d0a-bb90-b77f146f2b1b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.680692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a64b6e-c045-4d0a-bb90-b77f146f2b1b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.681199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a64b6e-c045-4d0a-bb90-b77f146f2b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.705589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-67a64b6e-c045-4d0a-bb90-b77f146f2b1b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:15.707313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67a64b6e-c045-4d0a-bb90-b77f146f2b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:15.832919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 139/273] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:15.886039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0235f73-c027-41e5-b34b-c3709bd33338 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.887037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0235f73-c027-41e5-b34b-c3709bd33338 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.887833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0235f73-c027-41e5-b34b-c3709bd33338 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.891090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 135/274] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:15.898799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217936c-4241-46ef-9aa6-31893d20c95e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:15.899869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217936c-4241-46ef-9aa6-31893d20c95e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:15.900681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217936c-4241-46ef-9aa6-31893d20c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:15.935019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7217936c-4241-46ef-9aa6-31893d20c95e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:15.937237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7217936c-4241-46ef-9aa6-31893d20c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:15.939826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0235f73-c027-41e5-b34b-c3709bd33338 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.015175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:16.093908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 140/275] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:16.114022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:16.145058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0235f73-c027-41e5-b34b-c3709bd33338 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.184337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217936c-4241-46ef-9aa6-31893d20c95e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.212079 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:16.213052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:16.213581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:16.214092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:16.234796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:16.246007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:16.247033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:16.247862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:16.248518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 136/276] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:16.258759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.328265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217936c-4241-46ef-9aa6-31893d20c95e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.376268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:16.377197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:16.377519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:16.377811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:16.392476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:16.398468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:16.399524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:16.400354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:16.400968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 141/277] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:16.407835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.447059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54073a7-8f9a-4fbc-90dd-cf56bb1ab16a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.466307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:16.505227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:16.506183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:16.506729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:16.507402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:16.508252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:16.508941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:16.511585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:16.533640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 137/278] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:16.535275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.554569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:16.566698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc4bba-5866-4a6d-ae2a-8711218f94df None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.616167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:16.616809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:16.617130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:16.617402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:16.621873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:16.621873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:16.621873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:16.630651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 142/279] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:16.631601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.688324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe5c4-b72f-42a5-8a5d-fd89b17e3ec7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.730230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:16.731257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:16.731530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:16.731938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] PATH_INFO: `/v3/users/29f5cdba84184c89ba98cabf060fc48f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:16.733979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] RBAC: Authorizing `identity:get_user(user_id=29f5cdba84184c89ba98cabf060fc48f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:16.736811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:16.741555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 138/280] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:16 2022] GET /identity/v3/users/29f5cdba84184c89ba98cabf060fc48f => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:16.741555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.758780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:16.774492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d2c47-547e-43c0-b38d-e6904c650305 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.813155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:16.813844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:16.814177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:16.814405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:16.820309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:16.822845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:16.825611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 143/281] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:16.826946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:16.849644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:16.897027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4b6cc-d82f-4b32-b94a-f63f81c82efa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.959148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:16.960202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:16.960653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:16.961115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] PATH_INFO: `/v3/users/9281f7d635c94d5cb565b7c207bd2b0b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:16.984484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f205b-150e-4c99-a958-c9c8c8780c56 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:16.991300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] RBAC: Authorizing `identity:get_user(user_id=9281f7d635c94d5cb565b7c207bd2b0b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:16.993345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:17.001100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 139/282] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:16 2022] GET /identity/v3/users/9281f7d635c94d5cb565b7c207bd2b0b => generated 293 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:17.001100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:17.044600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:17.045291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:17.045598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:17.045910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:17.047369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:17.049320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:17.055227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 144/283] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:16 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:17.055227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:17.091502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:17.144328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68868d57-8210-476f-928b-c6e2f7e172a7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:17.186147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1a738-612b-4c1f-a965-a831edb0ec5d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:17.188251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3dfd5-5a27-4b59-9be6-07698e96e162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:17.189018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3dfd5-5a27-4b59-9be6-07698e96e162 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:17.189304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3dfd5-5a27-4b59-9be6-07698e96e162 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:17.189569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3dfd5-5a27-4b59-9be6-07698e96e162 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:17.190375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3dfd5-5a27-4b59-9be6-07698e96e162 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:17.191855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3dfd5-5a27-4b59-9be6-07698e96e162 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:17.201118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:17.231604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 140/284] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:16 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:51:17.231604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44259d4-3729-4388-9431-16f920e639f2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:17.231604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44259d4-3729-4388-9431-16f920e639f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:17.231604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44259d4-3729-4388-9431-16f920e639f2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:17.231604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 141/285] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:17.231604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44259d4-3729-4388-9431-16f920e639f2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:17.237788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff46bf-e58d-4acc-bacc-99dc585cfc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:17.238325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff46bf-e58d-4acc-bacc-99dc585cfc06 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:17.238609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff46bf-e58d-4acc-bacc-99dc585cfc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:17.238886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff46bf-e58d-4acc-bacc-99dc585cfc06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:17.239690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff46bf-e58d-4acc-bacc-99dc585cfc06 None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, user.id=9281f7d635c94d5cb565b7c207bd2b0b, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:17.240006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff46bf-e58d-4acc-bacc-99dc585cfc06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:17.241520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff46bf-e58d-4acc-bacc-99dc585cfc06 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:17.247982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 145/286] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 12:51:17 2022] GET /identity/v3/role_assignments?user.id=9281f7d635c94d5cb565b7c207bd2b0b&scope.system=all&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:17.249298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e185-36ee-47c2-8ade-15444f76abc3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:17.249585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e185-36ee-47c2-8ade-15444f76abc3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:17.250020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e185-36ee-47c2-8ade-15444f76abc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:17.273279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ad19e185-36ee-47c2-8ade-15444f76abc3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:17.274679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ad19e185-36ee-47c2-8ade-15444f76abc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:17.409732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 146/287] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:17.470750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44259d4-3729-4388-9431-16f920e639f2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:17.523167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:17.523743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:17.524084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:17.524360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522/users/29f5cdba84184c89ba98cabf060fc48f/roles/b206ec3122d24117a73d7addacf3d732` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:17.534526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] RBAC: Authorizing `identity:create_grant(project_id=4de48b6c426840f4933ff94a73b59522, user_id=29f5cdba84184c89ba98cabf060fc48f, role_id=b206ec3122d24117a73d7addacf3d732)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:17.536174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:17.557908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 142/288] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:51:17 2022] PUT /identity/v3/projects/4de48b6c426840f4933ff94a73b59522/users/29f5cdba84184c89ba98cabf060fc48f/roles/b206ec3122d24117a73d7addacf3d732 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:17.755626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19e185-36ee-47c2-8ade-15444f76abc3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:17.784427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:17.886516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19e185-36ee-47c2-8ade-15444f76abc3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:17.956415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:17.957570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:17.958037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:17.958455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:17.992701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:18.002162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:18.003246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:18.004137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 147/289] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:18.011734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:18.114621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:18.208056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2db73-9879-41a1-bd50-c71b7227efbf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:18.275125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:18.276203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:18.276927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:18.277406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:18.278314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:18.278821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:18.280536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:18.303684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 143/290] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:18.312325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:18.457784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:18.489557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e9c1c-5d7e-4a56-a8ab-23741c54d1c5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:18.538935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:18.539666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:18.539982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:18.540251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] PATH_INFO: `/v3/users/65d7c478401d484cae6ca361a78ebc59` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:18.541596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] RBAC: Authorizing `identity:get_user(user_id=65d7c478401d484cae6ca361a78ebc59)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:18.543215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:18.545252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 148/291] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:18 2022] GET /identity/v3/users/65d7c478401d484cae6ca361a78ebc59 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:18.553096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:18.725420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c881a-0059-4af5-b00e-689f02e4d043 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:18.748809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:18.803422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:18.804978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:18.805738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:18.806471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:18.959455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:18.962712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:19.152380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 144/292] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:19.160926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:19.423485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ceefa1-57c8-4d95-974e-f9b6eaa1b746 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:19.501540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60e821-3a9e-4592-9abd-f8b6c578509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:19.502766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e821-3a9e-4592-9abd-f8b6c578509e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:19.503094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e821-3a9e-4592-9abd-f8b6c578509e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:19.503568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e821-3a9e-4592-9abd-f8b6c578509e None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:19.511243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60e821-3a9e-4592-9abd-f8b6c578509e None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:19.514043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60e821-3a9e-4592-9abd-f8b6c578509e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:19.517522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 149/293] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:19 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:19.529707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:19.564887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:19.758469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318a0bc-a48f-4c4a-ba7e-6e861a4e4f56 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:19.822424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d889744-e96c-4bca-ab3a-30ec0649179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:19.823129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889744-e96c-4bca-ab3a-30ec0649179b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:19.823409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889744-e96c-4bca-ab3a-30ec0649179b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:19.823681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889744-e96c-4bca-ab3a-30ec0649179b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:19.824437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d889744-e96c-4bca-ab3a-30ec0649179b None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=65d7c478401d484cae6ca361a78ebc59) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:19.824743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d889744-e96c-4bca-ab3a-30ec0649179b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:19.826487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d889744-e96c-4bca-ab3a-30ec0649179b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:19.836020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 145/294] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:19 2022] GET /identity/v3/role_assignments?user.id=65d7c478401d484cae6ca361a78ebc59&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:20.710736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc0731-553d-49fe-b686-2085c995db29 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:20.711717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc0731-553d-49fe-b686-2085c995db29 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:20.712152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc0731-553d-49fe-b686-2085c995db29 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:20.713970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 150/295] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:20.720871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd76991-d676-4893-9a9b-995e5881eade None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:20.722071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd76991-d676-4893-9a9b-995e5881eade None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:20.722562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd76991-d676-4893-9a9b-995e5881eade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:20.744585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5bd76991-d676-4893-9a9b-995e5881eade None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:20.746556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5bd76991-d676-4893-9a9b-995e5881eade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:20.747654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab773400-07ae-4382-84a4-9ef7535c3c2c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:20.747939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab773400-07ae-4382-84a4-9ef7535c3c2c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:20.748209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab773400-07ae-4382-84a4-9ef7535c3c2c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:20.750408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 151/296] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:51:20.756118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7901b30-b146-4cf7-af18-8aae24f2cb03 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:20.756723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7901b30-b146-4cf7-af18-8aae24f2cb03 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:20.756968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7901b30-b146-4cf7-af18-8aae24f2cb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:20.780392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b7901b30-b146-4cf7-af18-8aae24f2cb03 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:20.783010 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b7901b30-b146-4cf7-af18-8aae24f2cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:20.798786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:20.846336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 146/297] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:20.902491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 152/298] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:20.927941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfec3b-0ca8-46c1-8fff-4d2ee0548b77 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:20.928225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfec3b-0ca8-46c1-8fff-4d2ee0548b77 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:20.928469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfec3b-0ca8-46c1-8fff-4d2ee0548b77 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:20.930321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 147/299] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:20.936136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7c44-905f-4508-8bdd-d96e775dfde5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:20.936672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7c44-905f-4508-8bdd-d96e775dfde5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:20.937259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c7c44-905f-4508-8bdd-d96e775dfde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:20.958352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ce1c7c44-905f-4508-8bdd-d96e775dfde5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:20.959968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ce1c7c44-905f-4508-8bdd-d96e775dfde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:21.081306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 153/300] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:21.102398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec78a-aebc-4c5e-8ebb-d71fb5689c62 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.103131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec78a-aebc-4c5e-8ebb-d71fb5689c62 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.103882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec78a-aebc-4c5e-8ebb-d71fb5689c62 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.107142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 148/301] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:21.117208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f5352-4b30-4865-baf5-177004b00ffa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.118166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f5352-4b30-4865-baf5-177004b00ffa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.118905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f5352-4b30-4865-baf5-177004b00ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.132300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdec78a-aebc-4c5e-8ebb-d71fb5689c62 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.143259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:21.153831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-074f5352-4b30-4865-baf5-177004b00ffa None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:21.156381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-074f5352-4b30-4865-baf5-177004b00ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:21.255392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 154/302] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:21.256567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f5352-4b30-4865-baf5-177004b00ffa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.268822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:21.286776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdec78a-aebc-4c5e-8ebb-d71fb5689c62 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.345657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:21.347164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.347839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.348565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.369175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:21.385494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:21.386580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:21.387515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:21.390177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 149/303] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:21.395948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.424898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f5352-4b30-4865-baf5-177004b00ffa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.475766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:21.476415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.476691 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.476959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.488745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:21.493653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:21.494646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:21.495467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:21.499810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 155/304] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:21.500067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:21.500854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.555573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979c3cd-bf09-4dbb-b2bd-8c58cf9f21ae None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.583552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:21.595196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:21.595882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.596087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.596350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.597020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:21.597316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:21.598865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:21.608173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 150/305] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:21.609391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.624997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb33ab-5cdd-457c-b5ec-692c5aed9f3c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.665613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:21.666556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.666970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.667419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.685277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:21.690360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:21.691518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 156/306] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:21.692348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.751423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638cbbc-4be2-4112-828a-37ca901d97db None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.782306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:21.791597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:21.792225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.792509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.792785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.793466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:21.793867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:21.795225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:21.804563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 151/307] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:21.805466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.808285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d2b1-7656-4813-82c4-44c5dcbeb7a5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.842134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:21.852776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:21.853334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.853649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.853955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] PATH_INFO: `/v3/users/9281f7d635c94d5cb565b7c207bd2b0b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.855180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=9281f7d635c94d5cb565b7c207bd2b0b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:21.856668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:21.858325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 157/308] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/users/9281f7d635c94d5cb565b7c207bd2b0b => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:21.859196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:21.928319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8cca-d87f-490a-a29d-efe401ed6142 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.968578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:21.969212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:21.969487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:21.969793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:21.970459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:21.970786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:21.972150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:21.980082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce18df6-a88a-4b69-a1d4-1d537cfb7dd0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:21.981781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 152/309] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:21.982778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:22.037860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:22.038813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:22.039241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:22.039648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:22.040983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:22.042758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:22.044647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 158/310] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:22.045409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:22.064059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:22.125255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedc6ff-fba3-4f07-a661-d99f3d7d4473 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:22.139852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:22.175260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:22.175924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:22.176253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:22.176529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] PATH_INFO: `/v3/users/410801ceb0c84695bf8a0b72d555af1c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:22.196257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] RBAC: Authorizing `identity:get_user(user_id=410801ceb0c84695bf8a0b72d555af1c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:22.198112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:22.200235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 153/311] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:21 2022] GET /identity/v3/users/410801ceb0c84695bf8a0b72d555af1c => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:22.201365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:22.214154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48f9d8-f07d-403f-8367-f06df8b9fefd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:22.264411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:22.265448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:22.265740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:22.266029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] PATH_INFO: `/v3/system/users/9281f7d635c94d5cb565b7c207bd2b0b/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:22.267540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9281f7d635c94d5cb565b7c207bd2b0b, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:22.268936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:22.280826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 159/312] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Sep 7 12:51:22 2022] PUT /identity/v3/system/users/9281f7d635c94d5cb565b7c207bd2b0b/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:22.287266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:22.346637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10376a-244a-476f-84fe-62bf030e22c7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:22.367223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:22.412109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:22.412800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:22.413056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:22.413302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] PATH_INFO: `/v3/users/29f5cdba84184c89ba98cabf060fc48f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:22.415365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] RBAC: Authorizing `identity:get_user(user_id=29f5cdba84184c89ba98cabf060fc48f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:22.416881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:22.425267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 154/313] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:22 2022] GET /identity/v3/users/29f5cdba84184c89ba98cabf060fc48f => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:22.425267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:22.465056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9601c-7975-421b-b683-70f143d48cc4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:22.521334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:22.537802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:22.538853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:22.539437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:22.539954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] PATH_INFO: `/v3/groups` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:22.541220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:22.544106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:22.571716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 160/314] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Sep 7 12:51:22 2022] POST /identity/v3/groups => generated 229 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:51:22.573608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:22.598373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc86f68-be49-454e-b7e7-23db794f0a91 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:22.681068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:22.682756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:22.684133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:22.684825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:22.686783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] RBAC: Adding query filter params (role.id=b206ec3122d24117a73d7addacf3d732, user.id=410801ceb0c84695bf8a0b72d555af1c, scope.system=all) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:22.687669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:22.690281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:22.702794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 155/315] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 12:51:22 2022] GET /identity/v3/role_assignments?user.id=410801ceb0c84695bf8a0b72d555af1c&scope.system=all&role.id=b206ec3122d24117a73d7addacf3d732 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:22.772607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce75b-2cd8-4d9f-97a1-b6414801d608 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:22.820571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:22.822157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:22.822683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:22.823187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:22.891939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:22.910811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:22.912355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:23.081421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 161/316] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:23.091342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:23.172271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:23.281077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa18c58-e52c-4aa9-97e7-e74a06393f12 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:23.348476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2360f-7345-4389-b6b4-d311fa2a1edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:23.350064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2360f-7345-4389-b6b4-d311fa2a1edd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:23.350869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2360f-7345-4389-b6b4-d311fa2a1edd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:23.351656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2360f-7345-4389-b6b4-d311fa2a1edd None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:23.353964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2360f-7345-4389-b6b4-d311fa2a1edd None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:23.356287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2360f-7345-4389-b6b4-d311fa2a1edd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:23.360056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 156/317] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:23 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:23.369497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:23.551797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:23.580088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea120c7-0844-4fb3-a9a1-0d2297d43a8d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:23.637382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1c73e-4701-48df-bcb1-1d32e86c1b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:23.638376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1c73e-4701-48df-bcb1-1d32e86c1b18 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:23.638860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1c73e-4701-48df-bcb1-1d32e86c1b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:23.639334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1c73e-4701-48df-bcb1-1d32e86c1b18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:23.641730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1c73e-4701-48df-bcb1-1d32e86c1b18 None admin] RBAC: Adding query filter params (role.id=b206ec3122d24117a73d7addacf3d732, scope.project.id=4de48b6c426840f4933ff94a73b59522, user.id=29f5cdba84184c89ba98cabf060fc48f) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:23.642293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1c73e-4701-48df-bcb1-1d32e86c1b18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:23.645132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1c73e-4701-48df-bcb1-1d32e86c1b18 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:23.658676 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 162/318] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:51:23 2022] GET /identity/v3/role_assignments?user.id=29f5cdba84184c89ba98cabf060fc48f&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=b206ec3122d24117a73d7addacf3d732 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:24.725841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4af72-fe71-4605-adab-9ed74921d5d9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:24.726589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4af72-fe71-4605-adab-9ed74921d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:24.726690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4af72-fe71-4605-adab-9ed74921d5d9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:24.728489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 157/319] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:51:24.735251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604603ef-cc3c-442d-aebe-b80aaa332cb6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:24.735953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604603ef-cc3c-442d-aebe-b80aaa332cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:24.735953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604603ef-cc3c-442d-aebe-b80aaa332cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:24.758489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-604603ef-cc3c-442d-aebe-b80aaa332cb6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:24.759967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-604603ef-cc3c-442d-aebe-b80aaa332cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:24.846180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 163/320] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:25.111700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4af72-fe71-4605-adab-9ed74921d5d9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:25.235022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4af72-fe71-4605-adab-9ed74921d5d9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:25.249164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:25.281105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35debe-5cb9-4158-8c08-562710a87110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:25.281790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35debe-5cb9-4158-8c08-562710a87110 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.282135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35debe-5cb9-4158-8c08-562710a87110 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.282408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35debe-5cb9-4158-8c08-562710a87110 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.295130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35debe-5cb9-4158-8c08-562710a87110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0d35debe-5cb9-4158-8c08-562710a87110 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:25.301011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:25.303027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:25.305052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:25.313257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 158/321] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:25.313568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604603ef-cc3c-442d-aebe-b80aaa332cb6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:25.476820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:25.513478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604603ef-cc3c-442d-aebe-b80aaa332cb6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:25.564840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:25.565574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.565877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.566135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:25.567101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:25.568493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:25.578379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 164/322] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:25.580577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd4991-c12c-4439-8357-894156ba2ba1 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.581453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd4991-c12c-4439-8357-894156ba2ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.582232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd4991-c12c-4439-8357-894156ba2ba1 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.584022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:25.585489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 159/323] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:51:25.595284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccec06f-7b4c-4a25-a50a-18e379d67f7b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.595806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccec06f-7b4c-4a25-a50a-18e379d67f7b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.596273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccec06f-7b4c-4a25-a50a-18e379d67f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.617996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8ccec06f-7b4c-4a25-a50a-18e379d67f7b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:25.619406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8ccec06f-7b4c-4a25-a50a-18e379d67f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:25.702547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e7720-5409-4f14-9153-ede17f4b131d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:25.710206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 160/324] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:25.711500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7172127-f3c1-409f-b9f1-a4f4d95914ac None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.711854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7172127-f3c1-409f-b9f1-a4f4d95914ac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.712091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7172127-f3c1-409f-b9f1-a4f4d95914ac None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.714575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 161/325] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:25.726333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2b21-b239-499d-8465-ced702a72646 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.727360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2b21-b239-499d-8465-ced702a72646 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.727360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2b21-b239-499d-8465-ced702a72646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.756298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-50aa2b21-b239-499d-8465-ced702a72646 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:25.757658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-50aa2b21-b239-499d-8465-ced702a72646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:25.758217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:25.759329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.759781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.760553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] PATH_INFO: `/v3/users/9281f7d635c94d5cb565b7c207bd2b0b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.762828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] RBAC: Authorizing `identity:get_user(user_id=9281f7d635c94d5cb565b7c207bd2b0b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:25.764604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:25.774329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 165/326] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:25 2022] GET /identity/v3/users/9281f7d635c94d5cb565b7c207bd2b0b => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:25.775708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:25.796315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:25.885698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 162/327] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:25.940349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ef829-c6d3-4ca9-b9eb-8c04bcdacee4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:25.994058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:25.994563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:25.994840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:25.995110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:25.995838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] RBAC: Adding query filter params (role.id=ebf589306f5f4cbd8bc523efc556cd3d, user.id=9281f7d635c94d5cb565b7c207bd2b0b, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:25.996139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:25.997591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:26.012107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 166/328] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 12:51:25 2022] GET /identity/v3/role_assignments?user.id=9281f7d635c94d5cb565b7c207bd2b0b&scope.system=all&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:26.013977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2b21-b239-499d-8465-ced702a72646 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:26.052314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:26.141724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2b21-b239-499d-8465-ced702a72646 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:26.170132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:26.182116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:26.182776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:26.183060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:26.183335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:26.184535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:26.186099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:26.187995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 163/329] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:26.193836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:26.282088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:26.327306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:26.332665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7574394-7506-489f-ad5e-56d3d9a09ccf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:26.333606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e92f1-b1fc-4c00-9012-de60eb0fe537 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:26.374960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:26.375338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:26.375599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:26.375882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:26.376011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:26.376271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] PATH_INFO: `/v3/groups` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:26.376426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:26.376561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:26.376883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:26.378332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:26.393185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:26.406742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 164/330] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Sep 7 12:51:26 2022] POST /identity/v3/groups => generated 226 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:26.407031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:26.408627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:26.410193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 167/331] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:26.413869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:26.535205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e69ec-7553-48bd-a4e9-9f7664b5d690 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:26.574847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:26.575480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:26.575762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:26.576039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:26.576760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:26.577068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:26.578629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:26.588796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 165/332] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:26.594909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:26.707205 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a6e8c-ca6e-46e9-93de-178419cee149 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:26.752763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:26.766585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f39770-3b3f-427a-81f5-da7e80f5e81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:26.767635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f39770-3b3f-427a-81f5-da7e80f5e81d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:26.768123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f39770-3b3f-427a-81f5-da7e80f5e81d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:26.768587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f39770-3b3f-427a-81f5-da7e80f5e81d None admin] PATH_INFO: `/v3/users/410801ceb0c84695bf8a0b72d555af1c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:26.770597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f39770-3b3f-427a-81f5-da7e80f5e81d None admin] RBAC: Authorizing `identity:get_user(user_id=410801ceb0c84695bf8a0b72d555af1c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:26.773203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f39770-3b3f-427a-81f5-da7e80f5e81d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:26.776052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 168/333] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:26 2022] GET /identity/v3/users/410801ceb0c84695bf8a0b72d555af1c => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:26.782832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:26.815515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:26.957090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a169fbd-69da-46d8-ba97-aecfab1246af None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:27.017736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb64973-26b7-473d-8a5e-885161d7d983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:27.018825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb64973-26b7-473d-8a5e-885161d7d983 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:27.019301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb64973-26b7-473d-8a5e-885161d7d983 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:27.019766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb64973-26b7-473d-8a5e-885161d7d983 None admin] PATH_INFO: `/v3/system/users/410801ceb0c84695bf8a0b72d555af1c/roles/b206ec3122d24117a73d7addacf3d732` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:27.022275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb64973-26b7-473d-8a5e-885161d7d983 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=410801ceb0c84695bf8a0b72d555af1c, role_id=b206ec3122d24117a73d7addacf3d732)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:27.024930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb64973-26b7-473d-8a5e-885161d7d983 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:27.042142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 166/334] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Sep 7 12:51:26 2022] PUT /identity/v3/system/users/410801ceb0c84695bf8a0b72d555af1c/roles/b206ec3122d24117a73d7addacf3d732 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:29.720723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825ea2f-5fe1-4dcf-b802-6e3746ac690c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:29.721567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825ea2f-5fe1-4dcf-b802-6e3746ac690c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:29.721567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825ea2f-5fe1-4dcf-b802-6e3746ac690c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:29.722645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 169/335] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:29.732627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cd63d-30af-447d-b495-42d14ffefb6d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:29.733453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cd63d-30af-447d-b495-42d14ffefb6d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:29.733453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cd63d-30af-447d-b495-42d14ffefb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:29.775388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-454cd63d-30af-447d-b495-42d14ffefb6d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:29.777195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-454cd63d-30af-447d-b495-42d14ffefb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:29.879228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87ca91-8a95-4d08-af1c-d45d25acc8eb None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:29.879490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87ca91-8a95-4d08-af1c-d45d25acc8eb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:29.879863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87ca91-8a95-4d08-af1c-d45d25acc8eb None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:29.881386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 170/336] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:51:29.895384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f2aec-6441-4316-84ac-bb5559936749 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:29.895786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f2aec-6441-4316-84ac-bb5559936749 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:29.895940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f2aec-6441-4316-84ac-bb5559936749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:29.930440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 167/337] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:29.931259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fa0f2aec-6441-4316-84ac-bb5559936749 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:29.932680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa0f2aec-6441-4316-84ac-bb5559936749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:30.030712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 171/338] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:30.385748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cd63d-30af-447d-b495-42d14ffefb6d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:30.411999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:30.503111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f2aec-6441-4316-84ac-bb5559936749 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:30.544258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cd63d-30af-447d-b495-42d14ffefb6d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:30.617143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:30.617816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:30.618100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:30.618371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:30.631716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f2aec-6441-4316-84ac-bb5559936749 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:30.642319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:30.648581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:30.649583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:30.650763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:30.653416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 168/339] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:30.659872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:30.662307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:30.682489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:30.683141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:30.683414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:30.683680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:30.712649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:30.717969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:30.718926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:30.719755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 172/340] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:30.729414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:30.835571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:30.867719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e838e-0e69-4c43-89ec-d93d2c318f5a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:30.914569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6b377-a04a-4361-b656-fe02cab95a0c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:30.935660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:30.936602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:30.936980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:30.937257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:30.937994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:30.938300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:30.939903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:30.953917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:30.954618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:30.955014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:30.955343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 169/341] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:30.955449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:30.956112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:30.956416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:30.960670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:30.961424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:30.971230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 173/342] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:30.977083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:31.042123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:31.096657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7aa4c-2af1-4a20-813c-be69ab5f5562 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:31.136665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec747c7-55cb-41ae-ad3f-458a6d858520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:31.137292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec747c7-55cb-41ae-ad3f-458a6d858520 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.137567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec747c7-55cb-41ae-ad3f-458a6d858520 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.137870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec747c7-55cb-41ae-ad3f-458a6d858520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.171692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8c2c-820d-411c-995c-75d0d719ea62 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:31.215887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec747c7-55cb-41ae-ad3f-458a6d858520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:31.218020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec747c7-55cb-41ae-ad3f-458a6d858520 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:31.273165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:31.282092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:31.283116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.283387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.283775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.290788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:31.335769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 174/343] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:31.337097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dcc35-645d-40ca-80af-7fb006c59185 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.337376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dcc35-645d-40ca-80af-7fb006c59185 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.337649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dcc35-645d-40ca-80af-7fb006c59185 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.339098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 175/344] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:31.344853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dcc35-645d-40ca-80af-7fb006c59185 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:31.462180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dcc35-645d-40ca-80af-7fb006c59185 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:31.471995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:31.473861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:31.499756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba796-a2e7-46bb-8049-994dad1bf1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:31.500399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba796-a2e7-46bb-8049-994dad1bf1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.500734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba796-a2e7-46bb-8049-994dad1bf1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.501004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba796-a2e7-46bb-8049-994dad1bf1f6 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.502222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba796-a2e7-46bb-8049-994dad1bf1f6 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:31.503765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba796-a2e7-46bb-8049-994dad1bf1f6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:31.506075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 176/345] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:31 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:31.507263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac27c82-3e69-49ba-a7cc-55cf5574e4c7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.507537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac27c82-3e69-49ba-a7cc-55cf5574e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.507858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac27c82-3e69-49ba-a7cc-55cf5574e4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.533861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1ac27c82-3e69-49ba-a7cc-55cf5574e4c7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:31.535354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1ac27c82-3e69-49ba-a7cc-55cf5574e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:31.549271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:31.616648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 177/346] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:31.617532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac27c82-3e69-49ba-a7cc-55cf5574e4c7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:31.721379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 170/347] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:31.730166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:31.744143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac27c82-3e69-49ba-a7cc-55cf5574e4c7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:31.785713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:31.785713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.785713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.785713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.791468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:31.793003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:31.795520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 178/348] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:31 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:31.807035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:31.807566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:31.835681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:31.937642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df341dc-cbac-4e50-b75a-d096ae0e3d59 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:31.958322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128f4f-5506-4410-b870-cb62621fc684 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:31.976012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cefe68-03c7-4cc2-a1e7-d421328805b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:31.976529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cefe68-03c7-4cc2-a1e7-d421328805b3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.976876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cefe68-03c7-4cc2-a1e7-d421328805b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.977142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cefe68-03c7-4cc2-a1e7-d421328805b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.977936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cefe68-03c7-4cc2-a1e7-d421328805b3 None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=4de48b6c426840f4933ff94a73b59522) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:31.979116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cefe68-03c7-4cc2-a1e7-d421328805b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:31.979693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cefe68-03c7-4cc2-a1e7-d421328805b3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:31.986136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 179/349] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:31 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:31.987251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb5d3-5b36-4bce-a3b1-54093f1dd167 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.987545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb5d3-5b36-4bce-a3b1-54093f1dd167 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:31.987789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb5d3-5b36-4bce-a3b1-54093f1dd167 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:31.989001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 180/350] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:31.999568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba323a6-c44d-40a3-b654-0de2370558bb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:31.999958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba323a6-c44d-40a3-b654-0de2370558bb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.000118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba323a6-c44d-40a3-b654-0de2370558bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.034937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0ba323a6-c44d-40a3-b654-0de2370558bb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:32.038733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ba323a6-c44d-40a3-b654-0de2370558bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:32.051711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aa2eb-85f1-4f4d-b70a-0f40f81af924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:32.052388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa2eb-85f1-4f4d-b70a-0f40f81af924 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.052665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa2eb-85f1-4f4d-b70a-0f40f81af924 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.053026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa2eb-85f1-4f4d-b70a-0f40f81af924 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.057061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aa2eb-85f1-4f4d-b70a-0f40f81af924 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:32.060325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60aa2eb-85f1-4f4d-b70a-0f40f81af924 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:32.063860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 171/351] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:31 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:32.066189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e01fa-0db2-440b-ae27-419d5754d8f2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.066694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e01fa-0db2-440b-ae27-419d5754d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.067204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e01fa-0db2-440b-ae27-419d5754d8f2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.074546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 172/352] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:32.076384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e01fa-0db2-440b-ae27-419d5754d8f2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:32.192313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 181/353] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:32.195633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4740db-ad34-4c9f-a6c7-9a3066073e3c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.196748 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4740db-ad34-4c9f-a6c7-9a3066073e3c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.197561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4740db-ad34-4c9f-a6c7-9a3066073e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.212323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:32.230913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fe4740db-ad34-4c9f-a6c7-9a3066073e3c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:32.233073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fe4740db-ad34-4c9f-a6c7-9a3066073e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:32.339652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e01fa-0db2-440b-ae27-419d5754d8f2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:32.340988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 182/354] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:32.343569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:32.344618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4740db-ad34-4c9f-a6c7-9a3066073e3c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:32.419618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:32.420457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.420815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.421144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.422354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:32.424096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:32.426709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 173/355] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:32 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:32.434918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:32.501120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4740db-ad34-4c9f-a6c7-9a3066073e3c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:32.569067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:32.570140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.570635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.571104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.573565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:32.585102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:32.590155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60708b9f-98ec-456e-b702-e7e6e780d539 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:32.590942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:32.592360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:32.593939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:32.598395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 183/356] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:32.600187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:32.654670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:32.655486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.655690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.655957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.656752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=875a7b70c3d04438bc96aa0511c3c007) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:32.657061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:32.658661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:32.665346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 174/357] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:32 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:32.666264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:32.733887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495a69-4594-4310-a1b6-4e74ab91d28a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:32.791652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:32.794113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.794920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.795650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.804571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:32.813534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:32.821390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:32.823084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:51:32.824540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:32.829260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 184/358] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:32.837617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:32.920431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:32.926401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbdac5-5ea1-46b8-ac96-098f6d54d4ad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:32.981048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:32.983188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:32.983730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:32.984162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:32.985004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:32.985323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:32.987133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:33.008640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 175/359] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:33.015342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:33.031071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dfa97-7aeb-4e64-834d-daefcaf85bca None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:33.135421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:33.136322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:33.136769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:33.137275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:33.138498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:33.139005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:33.141492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:33.174923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 185/360] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:33.176879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:33.267899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:33.273866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06437dc-5b05-4e5c-9a10-5d9aeda91c86 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:33.343695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7e110-0ec3-449f-9127-0cb7618b5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:33.345232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7e110-0ec3-449f-9127-0cb7618b5dcc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:33.345837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7e110-0ec3-449f-9127-0cb7618b5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:33.346471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7e110-0ec3-449f-9127-0cb7618b5dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:33.439740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4134a-18ac-46cd-85fc-2707ddf94393 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:33.463667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:33.471229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7e110-0ec3-449f-9127-0cb7618b5dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:33.474020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7e110-0ec3-449f-9127-0cb7618b5dcc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:33.495026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:33.496214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:33.496701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:33.497184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:33.523509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:33.533009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:33.534088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:33.534950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:33.542433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 186/361] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:33.544134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:33.699333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:33.716027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 176/362] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:33.718754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f172ed-0087-406a-8b2a-1c354dd2b91f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:33.719775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f172ed-0087-406a-8b2a-1c354dd2b91f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:33.720506 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f172ed-0087-406a-8b2a-1c354dd2b91f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:33.723512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 177/363] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:33.725912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f172ed-0087-406a-8b2a-1c354dd2b91f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:33.760009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e175bf-1b1e-49a8-aced-1cf1c9c9dd69 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:33.831213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:33.832383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:33.832880 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:33.833399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] PATH_INFO: `/v3/users/410801ceb0c84695bf8a0b72d555af1c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:33.837872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] RBAC: Authorizing `identity:get_user(user_id=410801ceb0c84695bf8a0b72d555af1c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:33.840458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:33.845650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 187/364] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:51:33 2022] GET /identity/v3/users/410801ceb0c84695bf8a0b72d555af1c => generated 293 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:33.847150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:33.938123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:33.955698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f172ed-0087-406a-8b2a-1c354dd2b91f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:34.025913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4d9a0-7bea-4ac4-b34a-50e60c4c7030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:34.026939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4d9a0-7bea-4ac4-b34a-50e60c4c7030 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:34.027360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4d9a0-7bea-4ac4-b34a-50e60c4c7030 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:34.027827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4d9a0-7bea-4ac4-b34a-50e60c4c7030 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:34.029771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4d9a0-7bea-4ac4-b34a-50e60c4c7030 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:34.030295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4d9a0-7bea-4ac4-b34a-50e60c4c7030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:34.033153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4d9a0-7bea-4ac4-b34a-50e60c4c7030 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:34.052425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 178/365] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:34.058686 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410675f6-b5b4-4f17-9f4a-4488e611ada7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:34.059214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410675f6-b5b4-4f17-9f4a-4488e611ada7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:34.059786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410675f6-b5b4-4f17-9f4a-4488e611ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:34.066656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3d41b-fc86-423c-b73b-e367f437b66b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:34.090733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-410675f6-b5b4-4f17-9f4a-4488e611ada7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:34.092345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-410675f6-b5b4-4f17-9f4a-4488e611ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:34.154941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:34.155935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:34.156402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:34.156927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:34.159102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:34.161732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:34.164803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 188/366] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:33 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:34.170518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:34.244322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:34.269431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 179/367] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:34.271065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410675f6-b5b4-4f17-9f4a-4488e611ada7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:34.400154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac0e48-8c12-4b29-959d-ebdae8e5242f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:34.431103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:34.487507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:34.489993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:34.490761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:34.491905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:34.493557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] RBAC: Adding query filter params (role.id=b206ec3122d24117a73d7addacf3d732, user.id=410801ceb0c84695bf8a0b72d555af1c, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:34.494678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:34.498825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:34.520244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410675f6-b5b4-4f17-9f4a-4488e611ada7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:34.533904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 189/368] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 12:51:34 2022] GET /identity/v3/role_assignments?user.id=410801ceb0c84695bf8a0b72d555af1c&scope.system=all&role.id=b206ec3122d24117a73d7addacf3d732 => generated 510 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:34.535857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:34.598768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:34.599872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:34.600375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:34.600967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:34.724994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:34.741097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:34.742832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:34.855026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6094a0-14b7-4828-9f39-fae4eba8c5f2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:34.870040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 180/369] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:34.876281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:34.976366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:34.977438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:34.977958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:34.978447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:34.982942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:34.985503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:34.987656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:34.988569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 190/370] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:34 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:34.998803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4fc3d-9f92-4f2f-a511-07f15c592885 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:34.999088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:35.056510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:35.057717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:35.058225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:35.058759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:35.060595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:35.062367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:35.064823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 181/371] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:34 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:35.075030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:35.162355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:35.321782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61786d5c-f081-4b3c-80a3-deaebe80099d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:35.327214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345824-b5e9-4e6a-a7e3-496b5dc00bf2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:35.415136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:35.416151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:35.416648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:35.417140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:35.422241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:35.424948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:35.425225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:35.426233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:35.426717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:35.427188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:35.428251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=875a7b70c3d04438bc96aa0511c3c007) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:35.428378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 182/372] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:35 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:35.428881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:35.429741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:35.430790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:35.444740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 191/373] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:34 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 260 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:35.446204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:35.497502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:35.695967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215d17d-ba44-4bd5-b70f-974c560f7142 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:35.703532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:35.739803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef077-6c9a-4f6e-bcf8-a0707db3ec9b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:35.785710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:35.785710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:35.785710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:35.785710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:35.786222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=4de48b6c426840f4933ff94a73b59522) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:35.786222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:35.786687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:35.797431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 192/374] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:35 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 260 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:35.805953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c943f-b9f6-4758-a948-435dc63ba001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:35.807029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c943f-b9f6-4758-a948-435dc63ba001 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:35.807541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c943f-b9f6-4758-a948-435dc63ba001 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:35.808033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c943f-b9f6-4758-a948-435dc63ba001 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:35.826398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c943f-b9f6-4758-a948-435dc63ba001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d11c943f-b9f6-4758-a948-435dc63ba001 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:35.832322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:35.833334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:35.834261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:35.837519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 183/375] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:35.847403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:36.031971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddb260-9c61-47b8-b6ca-e71543634522 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:36.042488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4f25f-cccf-4972-9372-242ec45c1667 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:36.043293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4f25f-cccf-4972-9372-242ec45c1667 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:36.043783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4f25f-cccf-4972-9372-242ec45c1667 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:36.045975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 184/376] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:36.055328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced611e-5f09-4f30-b226-6d381cb5b862 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:36.055978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced611e-5f09-4f30-b226-6d381cb5b862 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:36.056455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced611e-5f09-4f30-b226-6d381cb5b862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:36.098896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dced611e-5f09-4f30-b226-6d381cb5b862 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:36.100996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dced611e-5f09-4f30-b226-6d381cb5b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:36.102251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:36.103214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:36.103870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:36.104544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:36.106017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:36.106841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:36.109747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:36.136232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 193/377] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:36.142802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:36.189065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:36.241574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 185/378] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:36.300541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962904b-1324-4c89-af1b-c5b0dc9b6cd3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:36.357997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:36.359012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:36.359497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:36.360053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:36.459737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852afa3-7b98-4935-9cd2-f1d0785a7d5d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:36.460255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852afa3-7b98-4935-9cd2-f1d0785a7d5d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:36.460719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852afa3-7b98-4935-9cd2-f1d0785a7d5d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:36.463301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 186/379] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:36.471617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:36.473403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:36.492665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee495-de8a-4cbf-9522-158d535d7189 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:36.492947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee495-de8a-4cbf-9522-158d535d7189 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:36.493220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee495-de8a-4cbf-9522-158d535d7189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:36.524109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1c4ee495-de8a-4cbf-9522-158d535d7189 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:36.527309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1c4ee495-de8a-4cbf-9522-158d535d7189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:36.528516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:36.555497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:36.632232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 194/380] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:36.645303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:36.704511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 187/381] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:36.705216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ee495-de8a-4cbf-9522-158d535d7189 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:36.820204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aab73-fd83-4163-a630-048a6d1099e4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:36.873992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:36.886743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:36.887410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:36.887913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:36.888463 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] PATH_INFO: `/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:36.889815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=f6924cc44bdf49bd840f84494b6ffd4d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:36.893721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:36.894137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 195/382] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:36 2022] GET /identity/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:36.913334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:37.021452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ee495-de8a-4cbf-9522-158d535d7189 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:37.111025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7780ee-ab08-4a3c-8a46-93b7f4088ef6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:37.154619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:37.155284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:37.155559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:37.155840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] PATH_INFO: `/v3/groups/68cbb64a4fd1440789bcba1b13d657eb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:37.161979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] RBAC: Authorizing `identity:get_group(group_id=68cbb64a4fd1440789bcba1b13d657eb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:37.163644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:37.164940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:37.165726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 196/383] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:36 2022] GET /identity/v3/groups/68cbb64a4fd1440789bcba1b13d657eb => generated 229 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:37.167108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:37.167828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:37.168560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:37.172149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:37.211177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:37.225703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:37.227375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:37.228877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:37.239954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 188/384] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 535 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:37.259325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:37.285633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:37.295273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:37.337604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b317-54ca-4392-8007-f00b831bbb67 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:37.393831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:37.393831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:37.393831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:37.393831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:37.394497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] RBAC: Adding query filter params (group.id=68cbb64a4fd1440789bcba1b13d657eb, role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=f6924cc44bdf49bd840f84494b6ffd4d) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:37.394497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:37.394497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:37.409705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 197/385] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:37 2022] GET /identity/v3/role_assignments?group.id=68cbb64a4fd1440789bcba1b13d657eb&scope.project.id=f6924cc44bdf49bd840f84494b6ffd4d&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:37.409705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:37.579383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950bd8f-96cf-4022-adc7-0e61933a2a8d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:37.597567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09694c-b607-4c43-8750-08d800274206 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:37.629056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:37.656925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:37.657926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:37.658415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:37.658897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:37.680468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:37.681553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:37.682083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:37.682795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:37.684049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:37.684617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:37.687593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:37.705513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 189/386] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:37.714026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:37.718855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:37.729465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:37.731148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:37.733416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:37.736207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 198/387] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:37.747375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:37.876238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:37.930156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36bf6f-d531-4994-bb55-80a361342bb3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:37.966024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a9d5-bbea-4088-8117-44a00cf853f0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:37.970296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:37.971057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:37.971337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:37.971608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:37.972289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:37.972592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:37.974570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:37.985235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 199/388] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:37.991232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:38.038529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:38.039203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:38.039487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:38.039770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:38.041064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:38.042890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:38.044817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 190/389] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:37 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:38.057716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:38.067580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:38.195565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f765-5ce3-4cc6-82d4-c6c86c846a51 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:38.286289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ad003-99ff-4784-ac34-a371ae3f057c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:38.329874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:38.330551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:38.330828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:38.331102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:38.343818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:38.343818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:38.343818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 191/390] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:38 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:38.358675 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:38.372381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:38.373483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:38.374019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:38.374583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:38.382527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:38.395244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:38.395493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:38.395653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 200/391] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:37 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:38.414929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:38.715164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6c3d-2025-4e42-b8a5-b746c6ae51ca None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:38.901523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:38.904979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:38.905436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:38.905892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:38.911853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] RBAC: Authorizing `identity:create_grant(project_id=4de48b6c426840f4933ff94a73b59522, group_id=fab6de66d3ab4f30bbe2b742c5ea0f24, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:38.914282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:38.956096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2d0ed-dfaa-4e28-81cf-ca0e888a9d83 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:38.963060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 192/392] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 12:51:38 2022] PUT /identity/v3/projects/4de48b6c426840f4933ff94a73b59522/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:39.002231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:39.098993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d9325-c83e-4247-9305-35635bf041b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:39.099976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d9325-c83e-4247-9305-35635bf041b2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:39.100394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d9325-c83e-4247-9305-35635bf041b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:39.100816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d9325-c83e-4247-9305-35635bf041b2 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:39.102928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d9325-c83e-4247-9305-35635bf041b2 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:39.105309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d9325-c83e-4247-9305-35635bf041b2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:39.117433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 201/393] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:38 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 704 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:39.127734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:39.294340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:39.343653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32efc7-2c0a-4b9b-b940-cf8b5203a797 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:39.393084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659c60c-5e6e-4159-8690-91162cf61439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:39.393821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c60c-5e6e-4159-8690-91162cf61439 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:39.394114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c60c-5e6e-4159-8690-91162cf61439 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:39.394376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c60c-5e6e-4159-8690-91162cf61439 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:39.399161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659c60c-5e6e-4159-8690-91162cf61439 None admin] RBAC: Authorizing `identity:create_grant(project_id=875a7b70c3d04438bc96aa0511c3c007, group_id=fab6de66d3ab4f30bbe2b742c5ea0f24, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:39.400604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659c60c-5e6e-4159-8690-91162cf61439 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:39.434299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 193/394] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 12:51:39 2022] PUT /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:39.598417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a782b-b0b4-47a4-8a3e-ab3c327f2e03 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:39.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a782b-b0b4-47a4-8a3e-ab3c327f2e03 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:39.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a782b-b0b4-47a4-8a3e-ab3c327f2e03 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:39.600264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 202/395] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:39.607407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98249aa9-2483-4fb8-b7be-f6c5c015b751 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:39.607870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98249aa9-2483-4fb8-b7be-f6c5c015b751 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:39.608331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98249aa9-2483-4fb8-b7be-f6c5c015b751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:39.647187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-98249aa9-2483-4fb8-b7be-f6c5c015b751 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:39.648521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-98249aa9-2483-4fb8-b7be-f6c5c015b751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:39.779207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 194/396] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:40.165750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a782b-b0b4-47a4-8a3e-ab3c327f2e03 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:40.323868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a782b-b0b4-47a4-8a3e-ab3c327f2e03 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:40.382242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:40.382912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:40.383249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:40.383530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:40.404369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:40.414612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:40.415672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:40.416566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 203/397] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:40.422538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98249aa9-2483-4fb8-b7be-f6c5c015b751 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:40.488442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:40.572166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98249aa9-2483-4fb8-b7be-f6c5c015b751 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:40.611339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:40.611979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:40.612264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:40.612699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:40.613614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:40.613962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:40.615436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:40.627203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 195/398] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:40.635589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:40.764512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:40.908270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f104be1-eb28-40d9-9fa7-ee71213e5b51 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:40.999576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191a46f-cf0f-428a-a78f-b60ccbf952a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:41.000584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191a46f-cf0f-428a-a78f-b60ccbf952a7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:41.001045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191a46f-cf0f-428a-a78f-b60ccbf952a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:41.001519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191a46f-cf0f-428a-a78f-b60ccbf952a7 None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:41.006816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191a46f-cf0f-428a-a78f-b60ccbf952a7 None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:41.009520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191a46f-cf0f-428a-a78f-b60ccbf952a7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:41.013797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 204/399] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:40 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:41.020156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:41.112545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf3db8-3e53-4846-acad-011e793efdd0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:41.113003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf3db8-3e53-4846-acad-011e793efdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:41.113490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf3db8-3e53-4846-acad-011e793efdd0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:41.115978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 205/400] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:41.127539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f818e4-6e6a-4aaf-93e8-4b2d83ead0ef None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:41.127856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f818e4-6e6a-4aaf-93e8-4b2d83ead0ef None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:41.128090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f818e4-6e6a-4aaf-93e8-4b2d83ead0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:41.165108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-83f818e4-6e6a-4aaf-93e8-4b2d83ead0ef None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:41.167164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-83f818e4-6e6a-4aaf-93e8-4b2d83ead0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:41.213049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f44c1-159e-4eae-a34c-48d2f12191c0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:41.237320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:41.261809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d7436-472a-4f3d-a951-131f4fa89f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:41.262900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d7436-472a-4f3d-a951-131f4fa89f05 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:41.263417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d7436-472a-4f3d-a951-131f4fa89f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:41.263888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d7436-472a-4f3d-a951-131f4fa89f05 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:41.265259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d7436-472a-4f3d-a951-131f4fa89f05 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:41.266939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d7436-472a-4f3d-a951-131f4fa89f05 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:41.273351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 196/401] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:41 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:41.273351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6119e1-b9eb-42e2-924c-9cdb0abbdc6c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:41.273351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6119e1-b9eb-42e2-924c-9cdb0abbdc6c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:41.273351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6119e1-b9eb-42e2-924c-9cdb0abbdc6c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:41.273351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 197/402] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:41.279080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6119e1-b9eb-42e2-924c-9cdb0abbdc6c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:41.302549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 206/403] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:41.311582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aea05-0605-4b27-95e6-21c4abdc9350 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:41.312055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aea05-0605-4b27-95e6-21c4abdc9350 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:41.312519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aea05-0605-4b27-95e6-21c4abdc9350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:41.360635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-300aea05-0605-4b27-95e6-21c4abdc9350 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:41.364233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-300aea05-0605-4b27-95e6-21c4abdc9350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:41.442433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:41.447977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6119e1-b9eb-42e2-924c-9cdb0abbdc6c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:41.518260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 207/404] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:41.568606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:41.569612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:41.570136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:41.570601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:41.580887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] RBAC: Authorizing `identity:create_grant(project_id=875a7b70c3d04438bc96aa0511c3c007, group_id=fab6de66d3ab4f30bbe2b742c5ea0f24, role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:41.583603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:41.615102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 198/405] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 12:51:41 2022] PUT /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:41.855629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300aea05-0605-4b27-95e6-21c4abdc9350 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:41.903566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:41.969709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300aea05-0605-4b27-95e6-21c4abdc9350 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:41.998410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:42.038633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:42.039295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.039569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.039837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.079454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:42.097056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:42.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:42.100882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:42.110677 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 208/406] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:42.114021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:42.184019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b5654-05a4-4f30-860e-5e9f7723443e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:42.231833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:42.322106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:42.326207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.327582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.328072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.359309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:42.378566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:42.380627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:42.382264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:42.385468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 199/407] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 478 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:42.402094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:42.425537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6789c-2d54-4759-9e78-8c394f79d8fe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:42.517253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:42.518499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.519222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.519695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.520903 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:42.521577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:42.524413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:42.553722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 209/408] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:42.560850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:42.582111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b44758-ad23-4091-a64b-62e138bbfa8c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:42.584841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:42.664937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:42.665479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.665972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.666250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.666930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:42.667235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:42.668765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:42.679940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 200/409] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:42.687026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:42.738225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:42.746823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069870f9-bf8c-4393-a089-d2d90702f28a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:42.787304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727ba3-f451-4b0f-9b27-c3caafdb64d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:42.789268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727ba3-f451-4b0f-9b27-c3caafdb64d5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.789545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727ba3-f451-4b0f-9b27-c3caafdb64d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.789906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727ba3-f451-4b0f-9b27-c3caafdb64d5 None admin] PATH_INFO: `/v3/groups/68cbb64a4fd1440789bcba1b13d657eb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.791540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727ba3-f451-4b0f-9b27-c3caafdb64d5 None admin] RBAC: Authorizing `identity:get_group(group_id=68cbb64a4fd1440789bcba1b13d657eb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:42.793802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727ba3-f451-4b0f-9b27-c3caafdb64d5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:42.796183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 210/410] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:42 2022] GET /identity/v3/groups/68cbb64a4fd1440789bcba1b13d657eb => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:42.797417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0c0c-0d5c-4929-bde8-5c7fb309d9c5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.797719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0c0c-0d5c-4929-bde8-5c7fb309d9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.797994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0c0c-0d5c-4929-bde8-5c7fb309d9c5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.800011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 211/411] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:42.805333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57421102-7469-43b2-beb1-cd2c49d94fdc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.805624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57421102-7469-43b2-beb1-cd2c49d94fdc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.805938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57421102-7469-43b2-beb1-cd2c49d94fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.831057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-57421102-7469-43b2-beb1-cd2c49d94fdc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:42.832272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57421102-7469-43b2-beb1-cd2c49d94fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:42.916609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32dffa-830d-470b-b2dd-de286aab3867 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:42.930266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 212/412] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:42.931698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57421102-7469-43b2-beb1-cd2c49d94fdc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:42.983627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e52cd-2112-4a3c-8f70-1d60ea28cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:42.984862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e52cd-2112-4a3c-8f70-1d60ea28cf4a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.985342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e52cd-2112-4a3c-8f70-1d60ea28cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.985830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e52cd-2112-4a3c-8f70-1d60ea28cf4a None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:42.988188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e52cd-2112-4a3c-8f70-1d60ea28cf4a None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:42.991271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e52cd-2112-4a3c-8f70-1d60ea28cf4a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:42.994738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 201/413] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:42 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:42.996263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd542c0-eddc-43c6-888e-3b0373b9bbf5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:42.997444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd542c0-eddc-43c6-888e-3b0373b9bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:42.998095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd542c0-eddc-43c6-888e-3b0373b9bbf5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.000474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 202/414] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:43.006381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd542c0-eddc-43c6-888e-3b0373b9bbf5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:43.071122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:43.075984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:43.089307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57421102-7469-43b2-beb1-cd2c49d94fdc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:43.154262 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2089-85e7-4259-b25b-a304bdb43cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:43.155158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2089-85e7-4259-b25b-a304bdb43cbb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:43.155584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2089-85e7-4259-b25b-a304bdb43cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:43.156050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2089-85e7-4259-b25b-a304bdb43cbb None admin] PATH_INFO: `/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.158221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd2089-85e7-4259-b25b-a304bdb43cbb None admin] RBAC: Authorizing `identity:get_project(project_id=f6924cc44bdf49bd840f84494b6ffd4d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:43.160375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd2089-85e7-4259-b25b-a304bdb43cbb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:43.162875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 213/415] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:42 2022] GET /identity/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d => generated 344 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:43.164717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8036229-0ccf-4007-bbdd-6bac9e8d5970 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:43.165236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8036229-0ccf-4007-bbdd-6bac9e8d5970 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:43.165688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8036229-0ccf-4007-bbdd-6bac9e8d5970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.189835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c8036229-0ccf-4007-bbdd-6bac9e8d5970 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:43.191418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c8036229-0ccf-4007-bbdd-6bac9e8d5970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:43.207350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd542c0-eddc-43c6-888e-3b0373b9bbf5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:43.257746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:43.258639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:43.259018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:43.259390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.260702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:43.262481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:43.269414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 203/416] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:43 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:43.270488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:43.285720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 214/417] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:43.285720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8036229-0ccf-4007-bbdd-6bac9e8d5970 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:43.437914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:43.455566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8036229-0ccf-4007-bbdd-6bac9e8d5970 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:43.480353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50645e-6c65-4e26-9eaa-25bcec44df3c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:43.482471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:43.520541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:43.521000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:43.521655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:43.522028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:43.522169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:43.522573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:43.522769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.523169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] PATH_INFO: `/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d/groups/68cbb64a4fd1440789bcba1b13d657eb/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.525313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] RBAC: Authorizing `identity:create_grant(project_id=f6924cc44bdf49bd840f84494b6ffd4d, group_id=68cbb64a4fd1440789bcba1b13d657eb, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:43.526336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] RBAC: Authorizing `identity:create_grant(project_id=4de48b6c426840f4933ff94a73b59522, group_id=fab6de66d3ab4f30bbe2b742c5ea0f24, role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:43.527160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:43.529084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:43.544866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 204/418] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 12:51:43 2022] PUT /identity/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d/groups/68cbb64a4fd1440789bcba1b13d657eb/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:43.546243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:43.551237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 215/419] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 12:51:43 2022] PUT /identity/v3/projects/4de48b6c426840f4933ff94a73b59522/groups/fab6de66d3ab4f30bbe2b742c5ea0f24/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:51:43.614784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:43.695972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa422-6243-48e5-913a-dd2348e0905c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:43.755886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:43.757741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:43.758751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:43.759800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.763028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423b74c-44b6-4275-a5b9-fb3a62216f9e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:43.785807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:43.799234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:43.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:43.802710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:43.810384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 205/420] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:43.821558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:43.867607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:43.868720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:43.869310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:43.869696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:43.888467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:43.895166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:43.897504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:43.899800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 216/421] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:43.904524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:43.906780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:43.932196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:44.165007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c5287-0fc4-4e87-a7ff-bc5bd57bc162 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:44.177048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702704fe-7855-4da3-b34d-3f529c7471fc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:44.234771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:44.236278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:44.236845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:44.237426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:44.238784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:44.239430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:44.242075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:44.256291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 217/422] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:44.265739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:44.291573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:44.292210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:44.292489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:44.292755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:44.294012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:44.294303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:44.295777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:44.314047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 206/423] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:51:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:44.320589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:44.454918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:44.478612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad648554-8299-4bed-8a67-ee3e2979e4a9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:44.531487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:44.532272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:44.532619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:44.532960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:44.547802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:44.613098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac66748-5a26-4b4a-97ba-bbe846454e42 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:44.636317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:44.638202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:44.693455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48153688-9da6-4dae-bc13-059f475aefed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:44.694228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48153688-9da6-4dae-bc13-059f475aefed None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:44.694571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48153688-9da6-4dae-bc13-059f475aefed None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:44.694979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48153688-9da6-4dae-bc13-059f475aefed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:44.768091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:44.801870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 207/424] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:44.811546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:44.906113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48153688-9da6-4dae-bc13-059f475aefed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:44.908042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48153688-9da6-4dae-bc13-059f475aefed None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:45.045827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eee0e-df4a-4396-bba0-ad1f60408255 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:45.111156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:45.112820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:45.113448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:45.113607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:45.114260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:45.117901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:45.121144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:45.137700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 208/425] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:44 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:45.137700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:45.186542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 218/426] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 922 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:45.189096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9e63c-6d57-4e2d-9995-ce1faa4ddb59 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:45.189721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9e63c-6d57-4e2d-9995-ce1faa4ddb59 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:45.190202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9e63c-6d57-4e2d-9995-ce1faa4ddb59 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:45.192787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 219/427] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:45.200112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9e63c-6d57-4e2d-9995-ce1faa4ddb59 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:45.243363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:45.332359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24e13-7306-4126-be6b-933b8cd60ed1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:45.333579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9e63c-6d57-4e2d-9995-ce1faa4ddb59 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:45.374708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2b786-c269-4a6f-8041-b939e45d31f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:45.381841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b786-c269-4a6f-8041-b939e45d31f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:45.382390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b786-c269-4a6f-8041-b939e45d31f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:45.382913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b786-c269-4a6f-8041-b939e45d31f9 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:45.384547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2b786-c269-4a6f-8041-b939e45d31f9 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:45.386476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2b786-c269-4a6f-8041-b939e45d31f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:45.389060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 220/428] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:45 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:45.390659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c275967-730c-4ac0-8d32-89b4587ec501 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:45.391189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c275967-730c-4ac0-8d32-89b4587ec501 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:45.391660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c275967-730c-4ac0-8d32-89b4587ec501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:45.400752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:45.403338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:45.403338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:45.403710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:45.405578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:45.408210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:45.417997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 209/429] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:45 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:45.419622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:45.422860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8c275967-730c-4ac0-8d32-89b4587ec501 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:45.424399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c275967-730c-4ac0-8d32-89b4587ec501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:45.467548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:45.513854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 221/430] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:45.515295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c275967-730c-4ac0-8d32-89b4587ec501 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:45.652319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c275967-730c-4ac0-8d32-89b4587ec501 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:45.676510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe78d7-d538-4460-ba74-8bc9adcc7fa1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:45.701550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:45.702565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:45.703056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:45.703618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:45.704761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=4de48b6c426840f4933ff94a73b59522) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:45.705348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:45.709428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:45.720950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 222/431] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:45 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:45.729434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:45.777745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:45.778762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:45.779200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:45.779634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:45.781757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:45.784286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:45.787630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 210/432] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:45 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:45.798922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:45.864808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:45.981695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec9541-63dc-4149-944b-e79af431fd03 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:46.026726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:46.027706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:46.028165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:46.028591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:46.029796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=ebf589306f5f4cbd8bc523efc556cd3d, scope.project.id=875a7b70c3d04438bc96aa0511c3c007) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:46.030272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:46.032637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:46.044924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 223/433] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:45 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=ebf589306f5f4cbd8bc523efc556cd3d => generated 611 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:46.170593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:46.351384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b7da5-b832-44de-b553-7e0dadfa98dc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:46.421808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:46.422805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:46.423183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:46.423463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:46.442881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:46.452428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:46.454253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:46.457021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 211/434] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:46.464166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:46.642678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:46.653405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ecf4-a876-424b-a69a-60127fec00dc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:46.706505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0e19-c240-4493-9a10-de1467e80979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:46.707395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0e19-c240-4493-9a10-de1467e80979 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:46.707886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0e19-c240-4493-9a10-de1467e80979 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:46.708299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0e19-c240-4493-9a10-de1467e80979 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:46.709107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0e19-c240-4493-9a10-de1467e80979 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:46.709553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0e19-c240-4493-9a10-de1467e80979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:46.711235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0e19-c240-4493-9a10-de1467e80979 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:46.722015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 224/435] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:46.728295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:46.817876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:46.915776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2dd4c-b70c-439b-a792-5e6df9189c3b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:46.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:46.958797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:46.959210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:46.959605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.040489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:47.042225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:47.094223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6dd08-71ca-407f-a684-578015dea9b9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.094595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6dd08-71ca-407f-a684-578015dea9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.094762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6dd08-71ca-407f-a684-578015dea9b9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.095243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:47.096481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 225/436] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:47.102450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ba9bf-3438-4655-87da-727caf732c6b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.102730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ba9bf-3438-4655-87da-727caf732c6b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.103004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ba9bf-3438-4655-87da-727caf732c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.123956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-459ba9bf-3438-4655-87da-727caf732c6b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:47.125335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-459ba9bf-3438-4655-87da-727caf732c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:47.167987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 212/437] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:51:47.169452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:47.199037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 226/438] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:51:47.246330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61172929-6f62-4545-bf71-26e9eaf312de None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.246669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61172929-6f62-4545-bf71-26e9eaf312de None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.247093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61172929-6f62-4545-bf71-26e9eaf312de None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.248488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 227/439] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:47.254563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb887cb6-9964-4df4-8306-c95e22ccec13 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.254902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb887cb6-9964-4df4-8306-c95e22ccec13 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.255179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb887cb6-9964-4df4-8306-c95e22ccec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.274206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fb887cb6-9964-4df4-8306-c95e22ccec13 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:47.275393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fb887cb6-9964-4df4-8306-c95e22ccec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:47.297160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ceaf3-78b2-40dc-8fcc-abcdc545650f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:47.338449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:47.339430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.340724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.341142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] PATH_INFO: `/v3/projects/875a7b70c3d04438bc96aa0511c3c007` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.342498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] RBAC: Authorizing `identity:get_project(project_id=875a7b70c3d04438bc96aa0511c3c007)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:47.344193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:47.346496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 213/440] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:47 2022] GET /identity/v3/projects/875a7b70c3d04438bc96aa0511c3c007 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:47.351711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 228/441] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:47.352686 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:47.384893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:47.471952 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb887cb6-9964-4df4-8306-c95e22ccec13 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:47.493688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:47.502295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9346f-2022-4d43-bd55-7cc844c319a8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:47.541235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:47.541914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.542195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.542466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.544470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:47.547207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:47.555159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 214/442] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:47 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:47.556812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:47.634805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb887cb6-9964-4df4-8306-c95e22ccec13 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:47.689885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:47.690806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.691284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.691644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.697902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:47.711413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:47.720153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:47.721895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:51:47.726787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:47.727432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bde71e-1e17-4e43-8ae2-4250817281e1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:47.727507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 229/443] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:51:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:47.732009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:47.777088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:47.777758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.778040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.778309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.779048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=875a7b70c3d04438bc96aa0511c3c007) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:47.779354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:47.780816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:47.790574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 215/444] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:47 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=875a7b70c3d04438bc96aa0511c3c007&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 611 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:47.792120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:47.881932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:47.918342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2302-2870-4017-9842-ffc92a488b79 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:47.987403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:47.988437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:47.988868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:47.989266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:47.990423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:47.990895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:47.992734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:47.995311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7a9e-b25e-4590-95f8-59d64b5d61c5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:48.005635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 230/445] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:51:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:48.023161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:48.071192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:48.072644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:48.073324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:48.073954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:48.095744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:48.107438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:48.108459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:48.109281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:48.109977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 216/446] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:48.120021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:48.135580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:48.205382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6a940-9567-495e-b8d4-56702a5a6d45 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:48.265404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:48.266807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:48.266807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:48.266807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:48.281301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:48.292991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de9909-5644-4d78-a1c2-b20bcc6ba16a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:48.352628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:48.353258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:48.353534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:48.353846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:48.354693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:48.355068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:48.357082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:48.369780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 217/447] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:48.380378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:48.410759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:48.413307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:48.515561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:48.564567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 231/448] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:48.570112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de197279-1139-4333-b3c2-82ccc6b7f9fb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:48.573074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:48.626175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:48.627721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:48.629270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:48.629781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:48.630265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:48.707672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7309b6f-fbe9-4d7a-a73f-d3a64cc73a35 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:48.743083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:48.745457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:48.764207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:48.765250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:48.765788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:48.766162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] PATH_INFO: `/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:48.769975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] RBAC: Authorizing `identity:get_project(project_id=f6924cc44bdf49bd840f84494b6ffd4d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:48.771727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:48.774343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 232/449] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:48 2022] GET /identity/v3/projects/f6924cc44bdf49bd840f84494b6ffd4d => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:48.782413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:48.858371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:48.914608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 218/450] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:51:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:51:48.915808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dadfe1-e642-44aa-b93d-75867896ada1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:48.920956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:48.928029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:48.959740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:48.960896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:48.961094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:48.961423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] PATH_INFO: `/v3/groups/68cbb64a4fd1440789bcba1b13d657eb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:48.962888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] RBAC: Authorizing `identity:get_group(group_id=68cbb64a4fd1440789bcba1b13d657eb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:48.964744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:48.967153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 233/451] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:48 2022] GET /identity/v3/groups/68cbb64a4fd1440789bcba1b13d657eb => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:48.973554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:49.055687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067b5f9-2b71-47a4-b7d9-9f015a6f0cec None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:49.101566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:49.102260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:49.102542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:49.102811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] PATH_INFO: `/v3/projects/4de48b6c426840f4933ff94a73b59522` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:49.104265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] RBAC: Authorizing `identity:get_project(project_id=4de48b6c426840f4933ff94a73b59522)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:49.106117 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:49.107960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 219/452] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:51:48 2022] GET /identity/v3/projects/4de48b6c426840f4933ff94a73b59522 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:49.113879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:49.120165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b541c-6a64-4385-9f2b-6bebacecaafe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:49.161215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:49.172286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:49.172920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:49.173200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:49.173552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:49.174294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] RBAC: Adding query filter params (group.id=68cbb64a4fd1440789bcba1b13d657eb, role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=f6924cc44bdf49bd840f84494b6ffd4d) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:49.174619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:49.176188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:49.182959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 234/453] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:48 2022] GET /identity/v3/role_assignments?group.id=68cbb64a4fd1440789bcba1b13d657eb&scope.project.id=f6924cc44bdf49bd840f84494b6ffd4d&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:49.265574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9178cc-97b9-481c-9754-f8b1e0aa94db None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:49.308850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebc7a2-1f2b-447f-85e5-3362a1612957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:49.309509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc7a2-1f2b-447f-85e5-3362a1612957 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:49.309891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc7a2-1f2b-447f-85e5-3362a1612957 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:49.310223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc7a2-1f2b-447f-85e5-3362a1612957 None admin] PATH_INFO: `/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:49.311604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebc7a2-1f2b-447f-85e5-3362a1612957 None admin] RBAC: Authorizing `identity:get_group(group_id=fab6de66d3ab4f30bbe2b742c5ea0f24)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:49.313243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebc7a2-1f2b-447f-85e5-3362a1612957 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:49.315195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 220/454] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 12:51:49 2022] GET /identity/v3/groups/fab6de66d3ab4f30bbe2b742c5ea0f24 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:49.322212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:49.379617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:49.465628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2aa38-4e8a-4342-9ce7-a0970822f5e6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:49.510268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ae319-2cd0-438c-8258-4b3525dcce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:49.510973 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae319-2cd0-438c-8258-4b3525dcce91 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:49.511746 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae319-2cd0-438c-8258-4b3525dcce91 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:49.512081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae319-2cd0-438c-8258-4b3525dcce91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:49.512901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae319-2cd0-438c-8258-4b3525dcce91 None admin] RBAC: Adding query filter params (group.id=fab6de66d3ab4f30bbe2b742c5ea0f24, role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=4de48b6c426840f4933ff94a73b59522) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:49.513323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae319-2cd0-438c-8258-4b3525dcce91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:49.515002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae319-2cd0-438c-8258-4b3525dcce91 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:49.522839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 235/455] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 12:51:49 2022] GET /identity/v3/role_assignments?group.id=fab6de66d3ab4f30bbe2b742c5ea0f24&scope.project.id=4de48b6c426840f4933ff94a73b59522&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:51:52.872213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3859e6-74c1-4969-81dc-30a8a6f0dd12 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:52.873774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3859e6-74c1-4969-81dc-30a8a6f0dd12 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:52.874500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3859e6-74c1-4969-81dc-30a8a6f0dd12 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:52.877662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 221/456] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:52.886353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e3477-d45b-4c59-8a02-bfd9da66cec4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:52.887942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e3477-d45b-4c59-8a02-bfd9da66cec4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:52.888701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e3477-d45b-4c59-8a02-bfd9da66cec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:52.926482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a73e3477-d45b-4c59-8a02-bfd9da66cec4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:52.929039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a73e3477-d45b-4c59-8a02-bfd9da66cec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:53.089393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 236/457] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:53.836676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3859e6-74c1-4969-81dc-30a8a6f0dd12 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:53.943024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d152-2de2-469d-b8c5-f3065f3b0123 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:53.944038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d152-2de2-469d-b8c5-f3065f3b0123 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:53.944533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d152-2de2-469d-b8c5-f3065f3b0123 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:53.946703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 237/458] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:53.953353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408103be-5543-43c8-8d2f-e8d4f9482bca None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:53.954045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408103be-5543-43c8-8d2f-e8d4f9482bca None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:53.954533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408103be-5543-43c8-8d2f-e8d4f9482bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:53.987760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-408103be-5543-43c8-8d2f-e8d4f9482bca None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:53.990120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-408103be-5543-43c8-8d2f-e8d4f9482bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:54.028440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:54.035785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3859e6-74c1-4969-81dc-30a8a6f0dd12 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:54.047866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:54.084715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:54.085924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.086403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.086966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.090767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 238/459] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:54.094331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7875db-5339-4dda-918f-dd47857be150 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.095092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7875db-5339-4dda-918f-dd47857be150 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.095790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7875db-5339-4dda-918f-dd47857be150 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.100753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 239/460] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:54.109912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4433-4276-43d2-8dbc-c481149c47b1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.109912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4433-4276-43d2-8dbc-c481149c47b1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.109912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4433-4276-43d2-8dbc-c481149c47b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.110759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:54.112258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:51:54.139434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:54.140519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:51:54.141454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:51:54.141454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:54.141454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:54.142445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 222/461] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:54.149727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:54.154435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f3eb4433-4276-43d2-8dbc-c481149c47b1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:54.155657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f3eb4433-4276-43d2-8dbc-c481149c47b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:54.312185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 240/462] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:54.347711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc0816-5421-4d9e-b0b4-303455ca381f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:54.409715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:54.410363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.410692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.410970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.411740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:54.412049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:54.413531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:54.421537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb4433-4276-43d2-8dbc-c481149c47b1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:54.432143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 223/463] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:54.441000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:54.478222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:54.537949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:54.547798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb4433-4276-43d2-8dbc-c481149c47b1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:54.565727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b162fc-8ec5-456b-8241-41f7744cef83 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:54.586058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e047b0c-c6fc-4312-9bb3-6ba7627605f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:54.587063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047b0c-c6fc-4312-9bb3-6ba7627605f3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.587550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047b0c-c6fc-4312-9bb3-6ba7627605f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.588029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e047b0c-c6fc-4312-9bb3-6ba7627605f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.599365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e047b0c-c6fc-4312-9bb3-6ba7627605f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:54.601158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e047b0c-c6fc-4312-9bb3-6ba7627605f3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:54.610705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef4792-e67b-49f9-a603-52873923561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:54.611333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4792-e67b-49f9-a603-52873923561d None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.611613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4792-e67b-49f9-a603-52873923561d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.611886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4792-e67b-49f9-a603-52873923561d None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.613082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef4792-e67b-49f9-a603-52873923561d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:54.614879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef4792-e67b-49f9-a603-52873923561d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9e047b0c-c6fc-4312-9bb3-6ba7627605f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:51:54.620349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:54.621728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:51:54.622607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:51:54.622607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:54.622607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:54.624841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 241/464] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:54.626439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f4b1b-4f0b-48ec-8065-b7242879e76c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.626952 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f4b1b-4f0b-48ec-8065-b7242879e76c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.627425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f4b1b-4f0b-48ec-8065-b7242879e76c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.629473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 242/465] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:54.631435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f4b1b-4f0b-48ec-8065-b7242879e76c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:54.652817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 224/466] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:54 2022] POST /identity/v3/users => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:51:54.654573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83539ee-a3e1-4c18-ab45-a54683c26ed6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.655149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83539ee-a3e1-4c18-ab45-a54683c26ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.655644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83539ee-a3e1-4c18-ab45-a54683c26ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.676659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a83539ee-a3e1-4c18-ab45-a54683c26ed6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:54.678383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a83539ee-a3e1-4c18-ab45-a54683c26ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:54.800431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f4b1b-4f0b-48ec-8065-b7242879e76c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:54.824062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 225/467] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:54.826667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83539ee-a3e1-4c18-ab45-a54683c26ed6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:54.833191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:54.875072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:54.875661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:54.875967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:54.876351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:54.877097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:54.877572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:54.879489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:54.902445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 243/468] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:54.908398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:54.920173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:55.010625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83539ee-a3e1-4c18-ab45-a54683c26ed6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:55.037774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4c490-e1ad-4cdf-88e6-3dfa5148f0a6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:55.053313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:55.054275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.054808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.055281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.068212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:55.070259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:55.077557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:55.078473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.078911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.079338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.080221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:55.081916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:51:55.088236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:55.089323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:51:55.090236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:51:55.090236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:55.090236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:55.090236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 226/469] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:55.096565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:55.111818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 244/470] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:54 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:51:55.180181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:55.237837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62640f8-fbc3-4bf8-b764-0be1605b836c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:55.277380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:55.278132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.278398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.278665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.279317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:55.279612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:55.281019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:55.295177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 227/471] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:55.303068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:55.353761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:55.420592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:55.489779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f70b4-052f-40b9-bd67-754bc7e06ee1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:55.503969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fd64c-9160-4940-84a5-1dfbd310951a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:55.527417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:55.539122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995c080-c631-4ea8-85c8-8eb9c72e763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:55.539759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c080-c631-4ea8-85c8-8eb9c72e763b None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.540041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c080-c631-4ea8-85c8-8eb9c72e763b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.540315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c080-c631-4ea8-85c8-8eb9c72e763b None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.540987 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995c080-c631-4ea8-85c8-8eb9c72e763b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:55.542563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995c080-c631-4ea8-85c8-8eb9c72e763b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:55.553876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:55.554791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.555246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.555661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.570894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:55.572809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:55.583502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 245/472] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:55 2022] POST /identity/v3/users => generated 248 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:51:55.587171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981559f-bf7c-42a7-97aa-9fb6ea0deaba None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.588032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981559f-bf7c-42a7-97aa-9fb6ea0deaba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.588866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981559f-bf7c-42a7-97aa-9fb6ea0deaba None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.593029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 246/473] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:51:55.593401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:55.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:51:55.596588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:51:55.596588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:55.596588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:55.596588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 228/474] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:55.599344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3b19d-8bf4-4e59-a410-83e4c9bc46be None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.599670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3b19d-8bf4-4e59-a410-83e4c9bc46be None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.599916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3b19d-8bf4-4e59-a410-83e4c9bc46be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.603671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:55.621599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2aa3b19d-8bf4-4e59-a410-83e4c9bc46be None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:55.622955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2aa3b19d-8bf4-4e59-a410-83e4c9bc46be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:55.735895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c7a46-7dd5-4afe-a612-9226b4839a30 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:55.753657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 247/475] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:55.783697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:55.784340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.784628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.784895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.785570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:55.785922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:55.787354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:55.805757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 229/476] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:55.807899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3b19d-8bf4-4e59-a410-83e4c9bc46be None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:55.872930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:55.938701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3b19d-8bf4-4e59-a410-83e4c9bc46be None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:55.979701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:55.980634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:55.980767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:55.981151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:55.981728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:55.983200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:56.011652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 248/477] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:51:55 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:51:56.088302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:56.125404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:56.219383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337f203-6b9a-4a5d-ab18-93ae1925f590 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:56.263154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:56.263701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:56.263979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:56.264256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:56.264960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:56.266433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:56.277291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.sql.core [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 12:51:56.277291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 12:51:56.277291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [parameters: {'id': '6282e5bf054f42b5aa746d2b6b02656e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 12:51:56.277291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81536) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Sep 07 12:51:56.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 12:51:56.296802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application yield resource Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise self.value Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application session.commit() Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 12:51:56.298655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.flush() Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application compat.raise_( Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 12:51:56.300339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 12:51:56.308779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 12:51:56.311811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [parameters: {'id': '6282e5bf054f42b5aa746d2b6b02656e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:56.314710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 12:51:56.317467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:56.326023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 230/478] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 12:51:56 2022] POST /identity/v3/roles => generated 144 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:56.326101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:56.454037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d863-2216-413e-8131-ad91c2b1faae None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:56.458074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:56.520394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a864bc7-ecbf-4f31-927b-f5c5b5b5a408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:56.521864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864bc7-ecbf-4f31-927b-f5c5b5b5a408 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:56.522434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864bc7-ecbf-4f31-927b-f5c5b5b5a408 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:56.523129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864bc7-ecbf-4f31-927b-f5c5b5b5a408 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:56.537868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a864bc7-ecbf-4f31-927b-f5c5b5b5a408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7a864bc7-ecbf-4f31-927b-f5c5b5b5a408 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:56.544710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:56.545760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:51:56.546598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:56.553932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 249/479] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:51:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:56.555560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:56.678080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:56.684516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534a2f2-1bce-4b6b-9483-9681272e2dd9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:56.727016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc5247-62c6-4248-9f62-2061288902a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:56.727939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc5247-62c6-4248-9f62-2061288902a7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:56.728310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc5247-62c6-4248-9f62-2061288902a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:56.728638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc5247-62c6-4248-9f62-2061288902a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:56.729605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc5247-62c6-4248-9f62-2061288902a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:56.730114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc5247-62c6-4248-9f62-2061288902a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:56.731766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc5247-62c6-4248-9f62-2061288902a7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:56.745824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 231/480] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:51:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:58.475294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476c90-7d0f-44d8-8992-9f350a13471d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:58.476492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476c90-7d0f-44d8-8992-9f350a13471d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:58.477050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476c90-7d0f-44d8-8992-9f350a13471d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:58.480562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 250/481] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:58.488933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682a64-aa80-4f80-aa80-5acb01173428 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:58.490236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682a64-aa80-4f80-aa80-5acb01173428 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:58.490842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682a64-aa80-4f80-aa80-5acb01173428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:58.519932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-54682a64-aa80-4f80-aa80-5acb01173428 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:58.521563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-54682a64-aa80-4f80-aa80-5acb01173428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:58.623084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 232/482] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:58.948775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14782d-6e7d-4cbd-95cb-3f23943afc88 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:58.949482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14782d-6e7d-4cbd-95cb-3f23943afc88 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:58.949981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14782d-6e7d-4cbd-95cb-3f23943afc88 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:58.952011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 251/483] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:58.957498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc9689-4042-4033-a816-a22ad7a2048e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:58.958130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc9689-4042-4033-a816-a22ad7a2048e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:58.958600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc9689-4042-4033-a816-a22ad7a2048e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:58.980629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3edc9689-4042-4033-a816-a22ad7a2048e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:58.982446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3edc9689-4042-4033-a816-a22ad7a2048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:51:59.075620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 233/484] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:51:59.102396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14782d-6e7d-4cbd-95cb-3f23943afc88 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:59.263678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14782d-6e7d-4cbd-95cb-3f23943afc88 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:59.329494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:59.331175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:59.331900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:59.332591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:59.353282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:59.363212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:59.364900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:51:59.366409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:59.371355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 252/485] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:59.379667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc9689-4042-4033-a816-a22ad7a2048e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:59.414084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:59.538500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:59.558817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc9689-4042-4033-a816-a22ad7a2048e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:59.631241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:59.650665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:59.651941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:59.652707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:59.653215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:59.654645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:51:59.655212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:59.658361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:59.678325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 234/486] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:51:59.685021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:59.824787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95dc86-a1a5-4fd4-9dde-9fc8415d0fee None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:59.834671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:51:59.866949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713b77-6fd6-4097-82c6-e8eda91cb8a3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:51:59.881011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa013e3-d4eb-4aa7-b5e3-de551f9d20c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:59.881719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa013e3-d4eb-4aa7-b5e3-de551f9d20c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:59.882006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa013e3-d4eb-4aa7-b5e3-de551f9d20c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:59.882279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa013e3-d4eb-4aa7-b5e3-de551f9d20c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:59.895490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa013e3-d4eb-4aa7-b5e3-de551f9d20c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8aa013e3-d4eb-4aa7-b5e3-de551f9d20c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:51:59.900565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:59.903165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:51:59.905030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 253/487] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:51:59.908216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6517-f6fc-4051-a2b0-10c3d342f90d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:59.908484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6517-f6fc-4051-a2b0-10c3d342f90d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:59.908763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6517-f6fc-4051-a2b0-10c3d342f90d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:59.911067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 254/488] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:59.916433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca599c-9b24-4646-a1fc-371b9742f3a2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:59.916769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca599c-9b24-4646-a1fc-371b9742f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:59.916984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca599c-9b24-4646-a1fc-371b9742f3a2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:59.918708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 255/489] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:51:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:51:59.919176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca599c-9b24-4646-a1fc-371b9742f3a2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:51:59.931843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f36c56-ea43-4a82-9f11-90d890fe9e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:51:59.932580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36c56-ea43-4a82-9f11-90d890fe9e0c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:59.932865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36c56-ea43-4a82-9f11-90d890fe9e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:59.933133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36c56-ea43-4a82-9f11-90d890fe9e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:59.948068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f36c56-ea43-4a82-9f11-90d890fe9e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:51:59.949603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f36c56-ea43-4a82-9f11-90d890fe9e0c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-11f36c56-ea43-4a82-9f11-90d890fe9e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:51:59.967884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:51:59.968893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:51:59.969816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:51:59.969816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:51:59.969816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:51:59.969816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 235/490] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:51:59.970555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385bec-0771-4bb3-8512-9881a93bd449 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:51:59.970653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385bec-0771-4bb3-8512-9881a93bd449 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:51:59.970915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385bec-0771-4bb3-8512-9881a93bd449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:51:59.995257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-84385bec-0771-4bb3-8512-9881a93bd449 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:51:59.996491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-84385bec-0771-4bb3-8512-9881a93bd449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:00.066947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:00.090348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca599c-9b24-4646-a1fc-371b9742f3a2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:00.105764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 236/491] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:00.105764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d530229-b015-4120-bf1a-e789d9001fb1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.105764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d530229-b015-4120-bf1a-e789d9001fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.105764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d530229-b015-4120-bf1a-e789d9001fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.125360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5d530229-b015-4120-bf1a-e789d9001fb1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:00.126868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d530229-b015-4120-bf1a-e789d9001fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:00.158027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:00.158990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.159468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.159931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.161091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:00.161618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:00.164173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:00.178857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 256/492] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:00.180303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:00.221443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:00.244244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 237/493] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:00.251376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4144b1-e0b2-49b1-8beb-2a7271a70e5e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.251692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4144b1-e0b2-49b1-8beb-2a7271a70e5e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.251912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4144b1-e0b2-49b1-8beb-2a7271a70e5e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.253571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 238/494] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:00.258004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4144b1-e0b2-49b1-8beb-2a7271a70e5e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:00.325219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0a334-039d-4432-944a-cadf50ab5ffb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:00.367994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1988a-b4a6-4edb-acf9-6d4bd7213a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:00.369372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1988a-b4a6-4edb-acf9-6d4bd7213a39 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.369925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1988a-b4a6-4edb-acf9-6d4bd7213a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.370396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1988a-b4a6-4edb-acf9-6d4bd7213a39 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.371352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1988a-b4a6-4edb-acf9-6d4bd7213a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:00.371847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1988a-b4a6-4edb-acf9-6d4bd7213a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:00.373904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1988a-b4a6-4edb-acf9-6d4bd7213a39 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:00.390409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 257/495] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:00.392375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1a191-5e83-4abd-8331-ceeac2e5d4c6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.392917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1a191-5e83-4abd-8331-ceeac2e5d4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.393479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1a191-5e83-4abd-8331-ceeac2e5d4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.399598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4144b1-e0b2-49b1-8beb-2a7271a70e5e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:00.412092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:00.423829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-01d1a191-5e83-4abd-8331-ceeac2e5d4c6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:00.425492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-01d1a191-5e83-4abd-8331-ceeac2e5d4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:00.451249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:00.452434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.452915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.453433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.467776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:00.470447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:00.490568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:00.491585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:00.492455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:00.492455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:00.492455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:00.495008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 239/496] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:00.496761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:00.529444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 258/497] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:00.530500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1a191-5e83-4abd-8331-ceeac2e5d4c6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:00.566560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:00.643317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df633d63-5537-4278-835e-bfbca31e0927 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:00.670048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1a191-5e83-4abd-8331-ceeac2e5d4c6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:00.684255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:00.685580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.686166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.686811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.696922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:00.699697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:00.710053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:00.710700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.710984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.711257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.711918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:00.712222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:00.713903 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:00.717849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:00.721036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:00.722718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 240/498] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:00.724385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:00.743042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 259/499] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:00.743042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:00.875542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e3d2f-e756-4594-9e58-011799813477 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:00.893157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:00.898658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec7d2c-35f8-4f28-b982-6e4668b59668 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:00.922800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:00.923501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.923777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.924383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.937456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:00.942794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:00.943804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:00.944614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 260/500] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:00.945304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:00.961830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:00.962486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:00.962803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:00.963473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:00.975754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:00.981738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:00.982699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:00.983527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 241/501] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:00.984214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.024283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.076629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08d962-b0da-4254-a8a1-76bf6438253c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.104292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda26ce-21f5-49ec-b96f-921aff0c0f4f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.130614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.131187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.131461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.131732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.132462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:01.132766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.134289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.143916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.144631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.144965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.145267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.155163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.156769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:01.171572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:01.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 242/502] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:01.175445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.183941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 261/503] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 12:52:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:01.184872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.206945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.303772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db72fb-1401-4e23-be23-7e24e76b73e3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.334858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffc64f-9e87-48af-89f2-1ebb9b623ab0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.340853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.341495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.341812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.342077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.353520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.354624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.355272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:01.375132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:01.376322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:01.377650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:01.377650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:01.377650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:01.377650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 243/504] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:01.378339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.386520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.387157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.387591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.388003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.389074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:01.389554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.391885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.405848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 262/505] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:01.406756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.478668 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.515209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c61b-8aa1-49ab-a9af-acaa6e5391cc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.523922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312120a-14d6-40db-a608-97ab4814a918 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.554145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.554828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.555104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.555398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.556146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:01.556446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.558271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.567367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.567990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.568276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.568549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.569288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:01.569574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.569786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 244/506] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:01.571159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.573906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.595597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 263/507] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:01.596968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.638834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.691376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b1ddf-618c-431e-b86b-9db6d7540016 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.710977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657abeb1-e5ca-4c39-bff1-345be16c1392 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.730014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.730835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.731202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.731745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.738436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.753747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.754376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.754644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.754907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.755609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:01.756692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.758397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.772438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 264/508] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:01.773509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.813049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.814598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.882667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.891964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5c64-e86f-4ba9-a085-6145e2c66bbb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:01.933446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 245/509] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:01.934442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:01.943345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:01.944046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:01.944410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:01.944768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:01.956753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:01.958280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:01.970765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:01.976006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:01.976962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:01.978278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:01.978278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:01.978278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:01.978278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 265/510] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:01.978278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.050697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548daf9-1432-4d3f-a75d-118adccd09ee None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:02.089268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:02.089951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:02.090131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30d915-4933-4e14-9d32-add9f2f50f0f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:02.090262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:02.090497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:02.107674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:02.109182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:02.126571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:02.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:02.130720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:02.130720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:02.130720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:02.130720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 246/511] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:02.130720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.136992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:02.137639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:02.137955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:02.138224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:02.159313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:02.227433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:02.229328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:02.230996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:02.243606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38140934-16fd-44ab-a44c-ba837317ec41 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:02.290678 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:02.291724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:02.292179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:02.292611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:02.293691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:02.296467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:02.327596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 247/512] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:52:02 2022] POST /identity/v3/users => generated 247 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:02.328702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.413209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 266/513] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:02.414920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.495021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:02.518619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd260aa-1818-42e6-8de9-715ac0019707 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:02.575874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:02.576911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:02.577356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:02.577836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:02.578321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f274463-62c4-4761-bae2-8b260fd4d5d2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:02.596610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:02.599013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:02.601526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:02.618636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:02.619744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:02.620611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:02.620611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:02.620611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:02.621830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 248/514] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:02.623669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.632460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:02.633099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:02.633391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:02.633714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:02.635018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:02.635018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:02.636305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:02.653695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 267/515] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:02.654807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.771735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f241abe-ed79-4b5f-92d1-04f472908307 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:02.789119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadef2d-79ed-47a7-9c27-1654052865d8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:02.799159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:02.833165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:02.834430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:02.835024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:02.835384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:02.835755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:02.836028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:02.836301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:02.836567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:02.837336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:02.838078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:02.840832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:02.849755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:02.851342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:02.867960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 249/516] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:02.867960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:02.874584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:02.875771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:02.876891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:02.876891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:02.876891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:02.882639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 268/517] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:02.882639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:02.948268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:03.008021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae784e9-2c5d-42bd-846c-d8347354d283 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.044233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcc4b1-3a46-4324-9e4a-e06083ebf2ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.049612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.050424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.050783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.051147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.051927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:03.052331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.053898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.071940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 250/518] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:03.071940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.107231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.108245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.108681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.109149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.121402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.123244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:03.140478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:03.142511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 12:52:03.144613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:03.150610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 269/519] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:03.152176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.177792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:03.218623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b0fad-52d7-43cc-93d1-898ee69b8498 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.239397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:03.275807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.276919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.277378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.277863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.290520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.293090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:03.309353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:03.311125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 251/520] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:03.312916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.324084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30a22a-92b9-44d3-9fba-3d4696255d5e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.367941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.368545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.368882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.369174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.369897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:03.370283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.371884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.398035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 270/521] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:03.398035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.481658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073ced2-3428-4f3a-be0c-70c09a5be212 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.524900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:03.527238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.528423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.528423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.528734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.536553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.538046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.552931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee67a80-8845-48aa-97e0-5f93255ee973 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:03.559876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:03.560844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:03.561753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:03.562223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 252/522] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:03.563574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.596777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.597400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.597700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.598086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.598912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:03.599220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.600810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.626600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 271/523] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:03.626600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.632778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:03.732524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7fae-769b-47de-8f82-0fbb17e42f91 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.784943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3b927-37f4-44b4-8bd6-3949cbed1646 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:03.793521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.794382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.794812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.795227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.796146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:03.796847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.799005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.836735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:03.837550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:03.837752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:03.837962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:03.842338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:03.849023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:03.850902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:03.858414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 253/524] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:03.859647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:03.867798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:03.870038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 272/525] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:03.880028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:03.988193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:04.001383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd13022-5d6a-4fdc-abb0-fd482acad5b4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:04.022856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b26a8-3efe-4f88-a5f0-73615e902fac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:04.048544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:04.049657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:04.049970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:04.050239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:04.050950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:04.051277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:04.053395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:04.071271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:04.072059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:04.072816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:04.073301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:04.079350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 254/526] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:04.079350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:04.200515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:04.212978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:04.215937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:04.219121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b805d-4b51-4487-8b1e-586651331709 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:04.273008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:04.273830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:04.274214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:04.274607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:04.308617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:04.354912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 273/527] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:04.355987 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:04.416933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:04.419268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:04.525882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:04.535018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f946ea-bf18-48a8-a804-a7af418af615 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:04.581550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 255/528] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:04.582286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:04.596583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:04.597449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:04.597991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:04.598370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:04.599455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:04.599967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:04.602138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:04.652519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 274/529] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:04.652519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:04.663910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:04.744501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18164c79-4ea4-4642-b388-f1e167ea710a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:04.796317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c888a44-956a-4376-9a36-00b7ac82a00e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:04.798365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:04.799024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:04.799314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:04.799592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:04.800371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=2aba60973c5e4aa7b0b680acb24d0431) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:04.800683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:04.802286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:04.810725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 256/530] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:04 2022] GET /identity/v3/role_assignments?user.id=2aba60973c5e4aa7b0b680acb24d0431&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:04.811620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:04.854104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:04.855146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:04.855607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:04.856057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:04.873049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:04.875414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:04.900666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:04.902908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:04.904611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:04.904611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:04.904611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:04.904611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 275/531] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:04.905048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:04.925181 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:04.999253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e262b-4a21-4dfa-88d5-9f04de1f617e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:05.026598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:05.050377 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd852c6-09ba-42ff-afb5-593e9d63364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:05.051224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd852c6-09ba-42ff-afb5-593e9d63364b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.051648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd852c6-09ba-42ff-afb5-593e9d63364b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.052065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd852c6-09ba-42ff-afb5-593e9d63364b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.070399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ee1e2-94e3-43bf-b9a7-60070a547b62 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:05.070676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd852c6-09ba-42ff-afb5-593e9d63364b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:05.073205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd852c6-09ba-42ff-afb5-593e9d63364b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4cd852c6-09ba-42ff-afb5-593e9d63364b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:05.099959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:05.101520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:05.103701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:05.103701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:05.103701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:05.103701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 257/532] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:05.103701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eae64-1e72-4750-bd27-6c1cebeb2c4d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.103701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eae64-1e72-4750-bd27-6c1cebeb2c4d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.104186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eae64-1e72-4750-bd27-6c1cebeb2c4d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.107081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 258/533] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:05.108082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eae64-1e72-4750-bd27-6c1cebeb2c4d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:05.132230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:05.133285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.133749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.134177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.135412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=bd45335f7a65416a8e2535c582f0624a) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:05.136095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:05.138090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:05.147993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 276/534] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:04 2022] GET /identity/v3/role_assignments?user.id=bd45335f7a65416a8e2535c582f0624a&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:05.151922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:05.284611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:05.306249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eae64-1e72-4750-bd27-6c1cebeb2c4d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:05.331229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fcfd4-8700-4344-9805-684a2678060f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:05.359928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b148-3af1-46b7-a729-e95cf093ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:05.361207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b148-3af1-46b7-a729-e95cf093ba36 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.361571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b148-3af1-46b7-a729-e95cf093ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.361946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b148-3af1-46b7-a729-e95cf093ba36 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.362991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b148-3af1-46b7-a729-e95cf093ba36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:05.363391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b148-3af1-46b7-a729-e95cf093ba36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:05.365766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b148-3af1-46b7-a729-e95cf093ba36 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:05.374694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:05.381661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.382242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.382664 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.383519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:05.383970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:05.385882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:05.388105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 259/535] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:05.390428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10d87-473a-4103-aa03-a6b6b7f077e6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.390873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10d87-473a-4103-aa03-a6b6b7f077e6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.391246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10d87-473a-4103-aa03-a6b6b7f077e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.407017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 277/536] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:05.408334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:05.429231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b1d10d87-473a-4103-aa03-a6b6b7f077e6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:05.432569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b1d10d87-473a-4103-aa03-a6b6b7f077e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:05.474227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:05.550880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 260/537] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:05.552671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d10d87-473a-4103-aa03-a6b6b7f077e6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:05.625115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6c78b-3708-4a37-b7d5-ede8a279f24b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:05.711625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:05.721762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:05.723867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.724558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.725213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.739391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:05.742563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:05.765909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:05.767505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:05.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:05.772893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 278/538] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:05.782902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:05.828530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d10d87-473a-4103-aa03-a6b6b7f077e6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:05.906183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:05.906824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:05.907100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:05.907369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:05.930907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:05.934126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:05.956158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:05.957101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:05.958001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 261/539] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:05.965824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:05.996554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b66ab9-68cf-46b8-a86a-9504c69eabad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:06.014746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:06.051235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:06.051958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:06.052305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:06.052646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:06.053501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:06.053943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:06.055768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:06.069890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 279/540] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:06.070993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:06.120060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8bda-98a5-4e30-832d-1e76dba5c0d3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:06.145550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:06.162816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:06.163443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:06.163792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:06.164132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:06.164911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:06.165284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:06.167039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:06.181023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 262/541] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:06.182131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:06.215891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d479-2d33-4661-8745-34d79ed46837 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:06.262036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:06.264237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:06.265317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:06.266253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:06.284811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:06.286562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:06.295709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:06.296679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:06.297878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:06.306289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 280/542] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:06.306289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:06.320436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74c90-53df-4ee0-a847-a3fd04b92b97 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:06.361874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:06.362817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:06.363258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:06.363677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:06.364646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=4302574365514eb1b25bd8e62736a4c5) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:06.365095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:06.366833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:06.374228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 263/543] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:06 2022] GET /identity/v3/role_assignments?user.id=4302574365514eb1b25bd8e62736a4c5&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:06.375524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:06.423858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbef0c3-0cba-4932-a4c2-71c2752f0eeb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:06.450589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:06.498471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:06.499355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:06.499740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:06.500113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:06.500969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=42f946fc56244e179973d6f846eb7548) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:06.501435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:06.503503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:06.515173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 281/544] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:06 2022] GET /identity/v3/role_assignments?user.id=42f946fc56244e179973d6f846eb7548&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:06.518398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dcc3b-341b-4941-b6b4-1629b281a225 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:06.574839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:06.575758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:06.576191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:06.576608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:06.577474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:06.577972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:06.579527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:06.590634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 264/545] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:06.596783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:06.762630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976094c-1a4c-4a4c-8d0b-81c72657c3e1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:06.799638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:06.800273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:06.800549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:06.800814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:06.813264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:06.814793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:06.830582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:06.831582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:06.832529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:06.832529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:06.832529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:06.832529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 282/546] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:06.839078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:06.846432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:07.011456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d26aa-9c21-45c0-a9a1-a59848275df3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:07.070695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe4f9f-8e4f-4bc3-bc70-940e1e0cd019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:07.071294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4f9f-8e4f-4bc3-bc70-940e1e0cd019 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.071574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4f9f-8e4f-4bc3-bc70-940e1e0cd019 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.071847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4f9f-8e4f-4bc3-bc70-940e1e0cd019 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.072784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe4f9f-8e4f-4bc3-bc70-940e1e0cd019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:07.073088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe4f9f-8e4f-4bc3-bc70-940e1e0cd019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:07.074633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe4f9f-8e4f-4bc3-bc70-940e1e0cd019 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:07.088856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 265/547] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:07.095328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:07.137288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:07.238456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc745ec-c016-4b24-988d-763859f6f3c5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:07.301036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:07.302117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.302624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.303101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.314794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:07.316688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:07.330078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:07.331277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:07.332231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:07.335166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 283/548] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:07.341712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593f1f6-c8ac-41af-8546-70e5e6412a17 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.341712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593f1f6-c8ac-41af-8546-70e5e6412a17 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.341712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593f1f6-c8ac-41af-8546-70e5e6412a17 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.341712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 266/549] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:07.343479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:07.347094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395463-a3aa-4adc-92d5-1f8722f49b0a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.347380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395463-a3aa-4adc-92d5-1f8722f49b0a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.347654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395463-a3aa-4adc-92d5-1f8722f49b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.376710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f2395463-a3aa-4adc-92d5-1f8722f49b0a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:07.380496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f2395463-a3aa-4adc-92d5-1f8722f49b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:07.528776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:07.548896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:07.560077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 267/550] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:52:07.586606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cdb7d-0bb2-42a3-8814-5726501523f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:07.644011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350981a-8950-4f07-81e3-f0b288534f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:07.644883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350981a-8950-4f07-81e3-f0b288534f0d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.645165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350981a-8950-4f07-81e3-f0b288534f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.645418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350981a-8950-4f07-81e3-f0b288534f0d None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.646503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350981a-8950-4f07-81e3-f0b288534f0d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:07.646880 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350981a-8950-4f07-81e3-f0b288534f0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:07.648599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350981a-8950-4f07-81e3-f0b288534f0d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:07.681778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 284/551] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 12:52:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:07.687859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395463-a3aa-4adc-92d5-1f8722f49b0a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:07.856387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395463-a3aa-4adc-92d5-1f8722f49b0a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:07.901493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:07.902199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.902514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.902690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c183670-6891-4eda-a0bf-27d8ae323323 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.902788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.902985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c183670-6891-4eda-a0bf-27d8ae323323 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.903264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c183670-6891-4eda-a0bf-27d8ae323323 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.904966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 285/552] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:52:07.911130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fb474-dfb6-426e-be2c-e3145f3b1de4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:07.911601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fb474-dfb6-426e-be2c-e3145f3b1de4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:07.912071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fb474-dfb6-426e-be2c-e3145f3b1de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:07.934340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-908fb474-dfb6-426e-be2c-e3145f3b1de4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:07.935591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-908fb474-dfb6-426e-be2c-e3145f3b1de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:08.013163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 286/553] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:08.055782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:08.070458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:08.072003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:08.100193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fb474-dfb6-426e-be2c-e3145f3b1de4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:08.235459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:08.247577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 268/554] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:08.256255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:08.261199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fb474-dfb6-426e-be2c-e3145f3b1de4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:08.300716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:08.301406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:08.301703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:08.301977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:08.315338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:08.320338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:08.321365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:08.322395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 287/555] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:08.337805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:08.442512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:08.450123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f7131-024d-4196-9961-96b9dc393767 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:08.472394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b714cb-d0a3-4014-b140-2bb12977b912 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:08.487687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:08.488865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:08.489356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:08.489862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:08.501177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:08.503526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:08.514485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:08.515154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:08.515438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:08.515711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:08.521758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:08.526500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:08.527994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:08.528991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:08.528991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:08.528991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:08.533104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 269/556] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:08.534350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:08.541455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:08.552093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:08.554270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:08.555584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 288/557] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:08.556972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:08.708358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56ee85-776f-4281-b70c-60d77463ffc8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:08.767153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:08.768865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:08.769527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:08.770545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:08.772491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:08.773512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:08.775319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c3828-8cee-456f-a690-ed2f33483287 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:08.776822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:08.805746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 270/558] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:08.807163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:08.829945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:08.840990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:08.841866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:08.842715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:08.843079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:08.843879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:08.844806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:08.846437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:08.872273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 289/559] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:08.872493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:08.945352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:08.952538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa418a-c543-43da-95c3-e8dde6075af9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:08.994401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:08.995576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:08.996039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:08.996472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:08.997357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:08.997900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:08.999613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.001751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f718ea-d1f9-49ee-a315-0a3da6322e9e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.011088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 271/560] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:09.012382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:09.048761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.049344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.049627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.049936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.063370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.065050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:09.083622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:09.084753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:09.085713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:09.085713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:09.085713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:09.090524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 290/561] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:09.091564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:09.106294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:09.147523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9395b-2db5-4123-b708-84dda40edd33 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.192987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.194514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.195240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.196136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.207425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.209480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:09.226702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:09.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:09.229357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:09.230989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ecafe-3f52-4ef8-ba13-3493015b89a9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.233607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 272/562] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:09.235454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:09.247851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:09.298674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.299864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.300306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.300807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.324652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.327767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:09.360214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:09.361835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:09.363172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:09.363172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:09.363172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:09.370271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 291/563] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:09.372173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:09.405076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693ce6-377a-49ae-b83a-7066c69dcdc6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.445633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:09.452276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.453265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.453786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.454299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.455244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:09.455609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.457134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.470393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 273/564] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:09.471300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:09.568488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54d5d2-ddf4-47c1-85e9-a273a1a101f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.598000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7425f1-2220-4caf-b1fc-a2ef7a393092 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.615569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f65ef-ef3a-4975-a55e-201465d7a3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.616200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f65ef-ef3a-4975-a55e-201465d7a3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.616476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f65ef-ef3a-4975-a55e-201465d7a3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.616749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f65ef-ef3a-4975-a55e-201465d7a3b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.617485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f65ef-ef3a-4975-a55e-201465d7a3b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:09.617834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f65ef-ef3a-4975-a55e-201465d7a3b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.619525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f65ef-ef3a-4975-a55e-201465d7a3b5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.634625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 292/565] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:09.635944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0849ad-488f-4c5c-9f4e-1e06fdfa599f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.636237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0849ad-488f-4c5c-9f4e-1e06fdfa599f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.636543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0849ad-488f-4c5c-9f4e-1e06fdfa599f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.637893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 293/566] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:09.638891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.639859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.640405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.640681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.641339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:09.641641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.643300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01675b4-00fe-48ca-991f-917872432ace None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.643530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.643666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01675b4-00fe-48ca-991f-917872432ace None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.643841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01675b4-00fe-48ca-991f-917872432ace None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.645193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 294/567] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:09.645958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01675b4-00fe-48ca-991f-917872432ace None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:09.660531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 274/568] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:09.661657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:09.677041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:09.732844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:09.774776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01675b4-00fe-48ca-991f-917872432ace None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.805969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c12b7-b3bc-4e56-abe7-a7406421ac2b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:09.824828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458e893-64b3-4722-acb6-fd17451f669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.825368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458e893-64b3-4722-acb6-fd17451f669c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.825646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458e893-64b3-4722-acb6-fd17451f669c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.825946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458e893-64b3-4722-acb6-fd17451f669c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.834864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458e893-64b3-4722-acb6-fd17451f669c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.836365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458e893-64b3-4722-acb6-fd17451f669c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2458e893-64b3-4722-acb6-fd17451f669c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:09.848756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:09.850499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 295/569] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532ec6-d491-49ce-91fd-9009acc85785 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.852149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532ec6-d491-49ce-91fd-9009acc85785 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.853533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532ec6-d491-49ce-91fd-9009acc85785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.858892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ddded-db22-459e-a6d1-6d1025ad7fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:09.859873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ddded-db22-459e-a6d1-6d1025ad7fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.860160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ddded-db22-459e-a6d1-6d1025ad7fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.860433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ddded-db22-459e-a6d1-6d1025ad7fe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.861180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ddded-db22-459e-a6d1-6d1025ad7fe9 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=30f70cdf08d049668d16007e4ca44654) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:09.861485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ddded-db22-459e-a6d1-6d1025ad7fe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:09.863080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ddded-db22-459e-a6d1-6d1025ad7fe9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:09.870337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 275/570] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:09 2022] GET /identity/v3/role_assignments?user.id=30f70cdf08d049668d16007e4ca44654&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:09.871619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e75d4-43c8-4f63-bd5f-14f35b714868 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:09.871934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e75d4-43c8-4f63-bd5f-14f35b714868 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:09.872182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e75d4-43c8-4f63-bd5f-14f35b714868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:09.875786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-16532ec6-d491-49ce-91fd-9009acc85785 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:09.877580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-16532ec6-d491-49ce-91fd-9009acc85785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:09.903965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c57e75d4-43c8-4f63-bd5f-14f35b714868 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:09.906004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c57e75d4-43c8-4f63-bd5f-14f35b714868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:10.013920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 296/571] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:10.015009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16532ec6-d491-49ce-91fd-9009acc85785 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.032982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:10.043880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 276/572] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:10.045928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e75d4-43c8-4f63-bd5f-14f35b714868 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.139850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:10.186304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16532ec6-d491-49ce-91fd-9009acc85785 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:10.240888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:10.242443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:10.242929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:10.243385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:10.253460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e75d4-43c8-4f63-bd5f-14f35b714868 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:10.262579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:10.264386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:10.289035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:10.291088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:10.292731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:10.296464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 297/573] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:10.305180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.315530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:10.316430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:10.316716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:10.316995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:10.318935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:10.319417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:10.321271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:10.342508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 277/574] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:10.354395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.452770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:10.487386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a67603-5959-4d99-b0f3-9acd2c8197a4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:10.489374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:10.494005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47be12-e775-4661-bbf1-a6315642707e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:10.528181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:10.528986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:10.529270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:10.529542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:10.544265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:10.546432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:10.554922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:10.557731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:10.558633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:10.560244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:10.562582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:10.563593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:10.566399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:10.566803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:10.568398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:10.570475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:10.570475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:10.570475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:10.570475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 278/575] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:10.570475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.610336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 298/576] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:10.612219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.729407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa2799-c879-4d76-bdaf-7ccfed05047e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:10.767683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaf136-d728-4844-a766-a7d1dd59f372 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:10.776944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:10.777703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:10.778084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:10.778438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:10.795295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:10.801872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:10.803140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:10.804315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:10.805133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:10.808177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 279/577] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:10.809148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.816198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:10.817072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:10.817502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:10.818067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:10.825964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:10.835600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:10.843969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:10.845169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:10.846270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:10.848749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 299/578] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:10.849906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:10.942528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c45772-bbd3-4304-85ab-3d3ff4ac7ad7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:10.984246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:10.984919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:10.985094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:10.985358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:10.986220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:10.986493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:10.987876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.002693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 280/579] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:11.003644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.020518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e156c-7448-47cc-ae4f-7f126b234bee None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.072355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.073004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.073282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.073557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.085264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.086812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:11.102447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:11.104164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:11.105789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:11.105789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:11.105789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:11.105789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 300/580] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:11.105789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.108738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:11.140251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04800d1-5a33-4eef-b071-cd7a0a32d4a8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.156038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:11.182080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.182745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.182998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.183275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.183939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:11.184253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.185641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.195276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 281/581] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:11.196761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.221002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95bbb-e147-4994-bec9-313af3325003 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.260403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.261391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.261391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.261629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.262137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:11.262443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.263858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.276684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 301/582] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:11.276684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.353807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8be4-e40e-41f6-bbbb-abba1c287c19 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.417373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.418219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.418726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.419100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.424574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:11.430765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.433305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:11.454861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:11.456704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:11.459160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:11.459972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce16f3-1fb6-4e8e-a794-ab667d30fb00 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.461020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:11.461876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 282/583] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:11.465658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.526786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.527624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.528108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.528645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.529650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:11.530184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.532796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.562914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 302/584] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:11.562914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.644489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ae6b9-8dde-4503-8e2c-88e7cd75e1b0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.698195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.698871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.699154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.699758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.713330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.715063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:11.733261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:11.743229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:11.745339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:11.745339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:11.745339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:11.745767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ed3ba-4ee0-4c2e-a685-2775b56bb366 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.745933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 283/585] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:11.745933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.805837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.807316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.807783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.808267 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.809960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:11.829957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.832776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:11.860788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:11.862018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:11.863202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:11.863202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:11.863202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:11.863837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 303/586] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:11.864917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:11.883615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f84f6b-5b65-48d1-889c-c3467b9a2162 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:11.897393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:11.934327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:11.934943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:11.935225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:11.935496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:11.936248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:11.936502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:11.937983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:11.953054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 284/587] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:11.953660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.020790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f320f97-8539-466c-a799-9ec21a2db88a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:12.068538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:12.069090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.069363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.070250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.077745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:12.079792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:12.083562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24cf3-66ae-45c4-898c-62d1da1df5da None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:12.097472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:12.098326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:12.099487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:12.100536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 304/588] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:12.102185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.136769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:12.137549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.137711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.138083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.138627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:12.138927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:12.140450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:12.159085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 285/589] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:12.159862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.238096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:12.285836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2916b21-8bd9-49e7-bb13-61dc87f0244d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:12.326857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a2f2d-a8e5-4cf8-843a-b60ec7bb9bc0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:12.348573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:12.349635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.349972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.350223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.351033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:12.351516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:12.353339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:12.367831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 305/590] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:12.371102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.391539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:12.392721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.393231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.393657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/2aba60973c5e4aa7b0b680acb24d0431/roles/bca79ab169d64049a6983bc348d0c161` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.424533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=2aba60973c5e4aa7b0b680acb24d0431, role_id=bca79ab169d64049a6983bc348d0c161)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:12.426949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:12.457803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 286/591] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:12 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/2aba60973c5e4aa7b0b680acb24d0431/roles/bca79ab169d64049a6983bc348d0c161 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:52:12.459298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.492144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:12.550879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aa5ae-d46c-497b-bdb9-25781f179eed None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:12.605954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:12.606912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.607290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.607779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.608816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:12.609171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:12.610758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:12.612937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:12.632299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 306/592] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:12.632299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.666412 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315ed17-4391-4b53-b1a2-bc24bd90c33e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:12.725247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3854bc6-b019-422d-943d-152f851ca79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:12.726344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3854bc6-b019-422d-943d-152f851ca79a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.726924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3854bc6-b019-422d-943d-152f851ca79a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.727418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3854bc6-b019-422d-943d-152f851ca79a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.742945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3854bc6-b019-422d-943d-152f851ca79a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:12.745813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3854bc6-b019-422d-943d-152f851ca79a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:12.763223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980676ac-2479-4b20-afc0-1fa98167ee7e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f3854bc6-b019-422d-943d-152f851ca79a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:12.769076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:12.770208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:52:12.771442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:12.773995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 287/593] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:12.776100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb17df-60ee-4256-b07c-76c66334414f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.776590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb17df-60ee-4256-b07c-76c66334414f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.777019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb17df-60ee-4256-b07c-76c66334414f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.778540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 288/594] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:12.779395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb17df-60ee-4256-b07c-76c66334414f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.835629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:12.836632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:12.836820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:12.837485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:12.847381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:12.848996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:12.864354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:12.865366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 307/595] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:12.867287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:12.921082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:12.949115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:12.962765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb17df-60ee-4256-b07c-76c66334414f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:13.010379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a0302-d54a-45d5-971f-ce3d39205ca2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:13.014942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8945d-095d-4492-bcfb-8a775b26fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:13.015696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8945d-095d-4492-bcfb-8a775b26fb05 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:13.016040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8945d-095d-4492-bcfb-8a775b26fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:13.016377 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8945d-095d-4492-bcfb-8a775b26fb05 None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/bd45335f7a65416a8e2535c582f0624a/roles/bca79ab169d64049a6983bc348d0c161` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:13.035628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8945d-095d-4492-bcfb-8a775b26fb05 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=bd45335f7a65416a8e2535c582f0624a, role_id=bca79ab169d64049a6983bc348d0c161)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:13.037432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8945d-095d-4492-bcfb-8a775b26fb05 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:13.061763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 289/596] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:12 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/bd45335f7a65416a8e2535c582f0624a/roles/bca79ab169d64049a6983bc348d0c161 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:52:13.061763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220c7af-b3b3-48cf-b63d-170e91a406bf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:13.061763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220c7af-b3b3-48cf-b63d-170e91a406bf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:13.061763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220c7af-b3b3-48cf-b63d-170e91a406bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:13.063930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:13.065020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:13.065591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:13.066141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:13.067502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:13.068173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:13.070737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:13.085467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3220c7af-b3b3-48cf-b63d-170e91a406bf None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:13.087450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3220c7af-b3b3-48cf-b63d-170e91a406bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:13.110060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 308/597] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 12:52:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:13.111231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:13.226366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 290/598] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:13.229076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220c7af-b3b3-48cf-b63d-170e91a406bf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:13.345310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8f3bd-1fa2-4869-81eb-e642fbad223b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:13.361478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:13.362449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:13.466784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220c7af-b3b3-48cf-b63d-170e91a406bf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:13.475927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:13.476632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:13.476901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:13.477177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:13.477980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:13.478988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:13.479794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:13.545061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:13.545502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 309/599] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Sep 7 12:52:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:13.547094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:13.547863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:13.548640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:13.555883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:13.578101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:13.581419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:13.621148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:13.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:13.624875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:13.624875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:13.624875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:13.631500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 291/600] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:13.642111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:13.822544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439dc02-3d79-4706-99e6-539b5a966912 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:13.844557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8392be-0b51-4fbf-8102-535f41e1cf6a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:13.863202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:13.881726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:13.883146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:13.883645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:13.884170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:13.897958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:13.899044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:13.899943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:13.900430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:13.902055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:13.902613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:13.904761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:13.907925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:13.911425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:13.929924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 292/601] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:13.929924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:13.939450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:13.940489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:13.941423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:13.941423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:13.941423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:13.941423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 310/602] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:13.942992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:13.977164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:14.073320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586c541-0958-4526-a82b-f9c5a2d3b804 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.082471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e511933-a91d-4f8b-8a1f-86a57d6b9479 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.127707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.128951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.129284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.129693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.132811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.133897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.134188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.134466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.143934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:14.144769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:14.145561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:14.151388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:14.152396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:14.153286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 293/603] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:14.155614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:14.164812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:14.166867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:14.167794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:14.172240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 311/604] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:14.173307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:14.218072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:14.301046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:14.321082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2af16-2f49-4924-ac31-a14bbaee226b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.343467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4ea8e-e42f-42b5-9c2f-3ba16fa9fcd2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.369684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.370321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.370603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.370874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.371530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:14.371831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:14.373646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:14.402367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 294/605] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:14.403659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:14.426500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.429130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.430005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.431596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.433481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:14.434719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:14.439635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:14.463535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 312/606] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:14.465060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:14.518491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:14.532686 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9871d-ac5b-4358-894b-bfed87b5703b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.571856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.572973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.573407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.573857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.574732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:14.575228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:14.577075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:14.584038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbb86a-dc79-4cd1-a618-8f81be47835b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.592442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 295/607] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:14.593634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:14.629467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.630598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.630773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.631150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.639531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:14.641187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:14.654075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:14.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:14.656411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:14.662628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 313/608] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:14.662628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:14.663638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:14.726919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed8fe-35f9-4a34-a306-7159644b9619 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.768135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.769095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.769523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.769966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.775061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc5010-58ea-4c69-b1c2-10c8221c33f1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:14.782435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:14.784242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:14.803165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:14.804192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:14.805111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:14.805111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:14.805111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:14.807736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 296/609] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:14.808925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:14.812301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:14.831194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:14.832012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:14.832356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:14.832683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/42f946fc56244e179973d6f846eb7548/roles/bca79ab169d64049a6983bc348d0c161` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:14.851118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=42f946fc56244e179973d6f846eb7548, role_id=bca79ab169d64049a6983bc348d0c161)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:14.852919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:14.873753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 314/610] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:14 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/42f946fc56244e179973d6f846eb7548/roles/bca79ab169d64049a6983bc348d0c161 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:52:14.875234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:15.011104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e8ca8-5fac-4b38-913b-97d5be8b5eb6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:15.024027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:15.111595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:15.112272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.112621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.112960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.114631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:15.115011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:15.116755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:15.135463 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e072645-ed40-49b5-bebe-a43988d0f38f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:15.149263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 297/611] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:15.157939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:15.212601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:15.214234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.214514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.214787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.215466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:15.215778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:15.217803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:15.242624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 315/612] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:15.242624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:15.278822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:15.330064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e6663-fed6-46e0-98c7-5a74c54fc665 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:15.363581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554043b-8f94-4897-a08f-8599395431fb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:15.375499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:15.387525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:15.388825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.389266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.389875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/4302574365514eb1b25bd8e62736a4c5/roles/bca79ab169d64049a6983bc348d0c161` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.405984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4a39c-84f5-4ec7-bb7f-bee1d926af32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:15.406775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4a39c-84f5-4ec7-bb7f-bee1d926af32 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.407158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4a39c-84f5-4ec7-bb7f-bee1d926af32 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.407554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4a39c-84f5-4ec7-bb7f-bee1d926af32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.413283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=4302574365514eb1b25bd8e62736a4c5, role_id=bca79ab169d64049a6983bc348d0c161)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:15.415287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:15.416301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4a39c-84f5-4ec7-bb7f-bee1d926af32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:15.417910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4a39c-84f5-4ec7-bb7f-bee1d926af32 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-46d4a39c-84f5-4ec7-bb7f-bee1d926af32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:15.430926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:15.432571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:15.433789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:15.439242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 316/613] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:15.440020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 298/614] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:15 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/4302574365514eb1b25bd8e62736a4c5/roles/bca79ab169d64049a6983bc348d0c161 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:52:15.440876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ca28c-83b7-4c8f-bbcc-900e42d9cc1e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.441379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ca28c-83b7-4c8f-bbcc-900e42d9cc1e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.441769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ca28c-83b7-4c8f-bbcc-900e42d9cc1e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.443526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 317/615] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:15.448349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:15.458960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8438952-8fc2-4135-bc0a-162522ccadd6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.458960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8438952-8fc2-4135-bc0a-162522ccadd6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.458960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8438952-8fc2-4135-bc0a-162522ccadd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.484573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d8438952-8fc2-4135-bc0a-162522ccadd6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:15.486622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d8438952-8fc2-4135-bc0a-162522ccadd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:15.648757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 318/616] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:15.662922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f8d4-da40-4f27-b7df-7662be49ccd4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:15.687505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:15.722224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:15.723184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.723521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.723852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.724881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:15.725267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:15.726894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:15.752648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 299/617] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 12:52:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:15.757654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8438952-8fc2-4135-bc0a-162522ccadd6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:15.903036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8438952-8fc2-4135-bc0a-162522ccadd6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:15.911116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:15.952006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad96f-44f9-42bd-9cd1-84495f2d451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:15.952641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad96f-44f9-42bd-9cd1-84495f2d451b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:15.952926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad96f-44f9-42bd-9cd1-84495f2d451b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:15.953199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad96f-44f9-42bd-9cd1-84495f2d451b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:15.965769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ad96f-44f9-42bd-9cd1-84495f2d451b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:15.967268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ad96f-44f9-42bd-9cd1-84495f2d451b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-922ad96f-44f9-42bd-9cd1-84495f2d451b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:15.988601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:15.990416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:15.992423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:15.992423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:15.992423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:15.992423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 319/618] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:15.997473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:16.012606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9ccf9-34ee-41ad-8785-f8ec8b80bdf4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:16.013008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9ccf9-34ee-41ad-8785-f8ec8b80bdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:16.013263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9ccf9-34ee-41ad-8785-f8ec8b80bdf4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:16.015770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 320/619] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:52:16.021422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930122-304c-47b3-9266-5a2ea54e9942 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:16.021838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930122-304c-47b3-9266-5a2ea54e9942 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:16.022104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930122-304c-47b3-9266-5a2ea54e9942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:16.043877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-34930122-304c-47b3-9266-5a2ea54e9942 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:16.045108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34930122-304c-47b3-9266-5a2ea54e9942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:16.188481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 321/620] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:52:16.188751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57a37a-ee63-4264-b0d9-4176f2c0e8c5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:16.215749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34930122-304c-47b3-9266-5a2ea54e9942 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:16.250108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:16.250742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:16.251033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:16.251386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:16.252335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:16.252641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:16.254088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:16.273525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 300/621] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:16.280328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:16.322896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:16.499385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:16.527756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34930122-304c-47b3-9266-5a2ea54e9942 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:16.548880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9941b-969e-4253-a0cc-d558743b4b7c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:16.589547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:16.590650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:16.591132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:16.591596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:16.611935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:16.621191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:16.623149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:16.624343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 322/622] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:16.632742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:16.651993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:16.652933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:16.653406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:16.653930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:16.663594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:16.665203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:16.679364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:16.682302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:16.684526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:16.689926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 301/623] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:16.692074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:16.800573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215f534-0e23-4a7e-85e8-07e399afae20 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:16.824697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:16.838504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9a997-ffa4-4ad8-8150-5a5c74345d63 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:16.864451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856788ff-345c-486d-89c8-4978982d323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:16.865959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856788ff-345c-486d-89c8-4978982d323e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:16.866713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856788ff-345c-486d-89c8-4978982d323e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:16.867464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856788ff-345c-486d-89c8-4978982d323e None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:16.870015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856788ff-345c-486d-89c8-4978982d323e None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:16.870868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856788ff-345c-486d-89c8-4978982d323e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:16.873961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856788ff-345c-486d-89c8-4978982d323e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:16.886043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:16.886669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:16.886952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:16.887227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:16.892263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:16.892263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:16.892263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:16.893086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 323/624] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:16.894337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856788ff-345c-486d-89c8-4978982d323e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:16.910648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 302/625] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:16.912091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:16.915395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:17.053198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856788ff-345c-486d-89c8-4978982d323e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:17.077832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fb7d7-5e2f-4380-a886-187f483c7954 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:17.092716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:17.093629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:17.094189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:17.095058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:17.108509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:17.115297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:17.116445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:17.117556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:17.124849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 324/626] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:17.124849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:17.146297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:17.147415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:17.147950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:17.148452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:17.167167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:17.169827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:17.198654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:17.199698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:17.200729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:17.200729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:17.200729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:17.201405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 303/627] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:17.202466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:17.208889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:17.222240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:17.254377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1faa8b-18da-43d8-8fcd-68dad1d8bba6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:17.296875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:17.297859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:17.298328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:17.298775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/30f70cdf08d049668d16007e4ca44654/roles/bca79ab169d64049a6983bc348d0c161` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:17.320745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=30f70cdf08d049668d16007e4ca44654, role_id=bca79ab169d64049a6983bc348d0c161)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:17.323859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:17.348807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 325/628] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:17 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/30f70cdf08d049668d16007e4ca44654/roles/bca79ab169d64049a6983bc348d0c161 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:52:17.349943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:17.394869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f165688-88d1-4213-b5f7-a99662a4e0b6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:17.491568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:17.492485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:17.492835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:17.493284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:17.494081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:17.494449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:17.496572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:17.524940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 304/629] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:17.525776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229133-3806-4b2e-acbb-458c50bbb9ae None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:17.536602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:17.550923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:17.572047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:17.573033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:17.573855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:17.574331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:17.575718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:17.576208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:17.578116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:17.594618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 326/630] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:17.599820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:17.652381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:17.715488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c098c-9a6c-479a-835d-d483ab8f1135 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:17.779972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07033d9-ac91-46d7-8b24-9c4b9c17bf79 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:17.780685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:17.781650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:17.782092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:17.782486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:17.797081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:17.798900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:17.825175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:17.826648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:17.828075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:17.828075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:17.828075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:17.833699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 305/631] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:17.837739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:17.843982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:17.845086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:17.845621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:17.846121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:17.877430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:17.879329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:17.902848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:17.904083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:17.906423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 327/632] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:17.913542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:18.018433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:18.122895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36ba62-9ed0-4c53-a1e1-9ebe4cc49b4b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:18.154024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf3430-c288-45b6-ba95-c70ac8dea519 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:18.164966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:18.206913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:18.208500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.209337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.210107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.212076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:18.212988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:18.216092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:18.240831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 328/633] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 12:52:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:18.246830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:18.254525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:18.255212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.255499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.255778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.256458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:18.256768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:18.258329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:18.289874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 306/634] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:18.297382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:18.378306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:18.387182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0ad87-799e-4b30-b4bb-17c3dbe42620 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:18.427464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9968c5-b121-4226-88cd-7523e74ca57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:18.428266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9968c5-b121-4226-88cd-7523e74ca57e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.428603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9968c5-b121-4226-88cd-7523e74ca57e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.428998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9968c5-b121-4226-88cd-7523e74ca57e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.526078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9968c5-b121-4226-88cd-7523e74ca57e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:18.528859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9968c5-b121-4226-88cd-7523e74ca57e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:18.582696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68838-e1cd-4103-bff1-81fafd52c1ce None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:18.633865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f9e79-7889-48df-9b5e-ed8715e0f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:18.635760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f9e79-7889-48df-9b5e-ed8715e0f03a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.636632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f9e79-7889-48df-9b5e-ed8715e0f03a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.637441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f9e79-7889-48df-9b5e-ed8715e0f03a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.653239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f9e79-7889-48df-9b5e-ed8715e0f03a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:18.658229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f9e79-7889-48df-9b5e-ed8715e0f03a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4b3f9e79-7889-48df-9b5e-ed8715e0f03a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:18.679183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:18.680873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:18.682515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:18.684819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 307/635] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:52:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:18.685158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208ecd-1bbb-43c9-a214-b570b4b0d6b8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.685662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208ecd-1bbb-43c9-a214-b570b4b0d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.686994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208ecd-1bbb-43c9-a214-b570b4b0d6b8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.688271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 308/636] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:18.694841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208ecd-1bbb-43c9-a214-b570b4b0d6b8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:18.727944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:18.740189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 329/637] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:52:18.742314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f884d5b-7617-4984-9e99-76ff812e8334 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.743276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f884d5b-7617-4984-9e99-76ff812e8334 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.743743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f884d5b-7617-4984-9e99-76ff812e8334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.749961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:18.778944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2f884d5b-7617-4984-9e99-76ff812e8334 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:18.780764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2f884d5b-7617-4984-9e99-76ff812e8334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:18.866764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208ecd-1bbb-43c9-a214-b570b4b0d6b8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:18.901702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 330/638] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:18.903743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f884d5b-7617-4984-9e99-76ff812e8334 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:18.922462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56d16d-98ca-478e-9fb5-ad4c967d2566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:18.923158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d16d-98ca-478e-9fb5-ad4c967d2566 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.923442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d16d-98ca-478e-9fb5-ad4c967d2566 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.923711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d16d-98ca-478e-9fb5-ad4c967d2566 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.924455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56d16d-98ca-478e-9fb5-ad4c967d2566 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:18.924763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56d16d-98ca-478e-9fb5-ad4c967d2566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:18.926480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56d16d-98ca-478e-9fb5-ad4c967d2566 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:18.948592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 309/639] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:52:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:18.950057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026051d6-3cd7-452e-a813-5ce0872b5df0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:18.950346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026051d6-3cd7-452e-a813-5ce0872b5df0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:18.950615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026051d6-3cd7-452e-a813-5ce0872b5df0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:18.956408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 310/640] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:18.957347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026051d6-3cd7-452e-a813-5ce0872b5df0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:19.100862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:19.106945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026051d6-3cd7-452e-a813-5ce0872b5df0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.109368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f884d5b-7617-4984-9e99-76ff812e8334 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.158890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:19.170629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:19.171618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.172128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.172592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:19.175090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749db41-7262-4854-b701-930f85695c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:19.176098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749db41-7262-4854-b701-930f85695c45 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.176508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749db41-7262-4854-b701-930f85695c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.176978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749db41-7262-4854-b701-930f85695c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:19.195364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749db41-7262-4854-b701-930f85695c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:19.198301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749db41-7262-4854-b701-930f85695c45 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8749db41-7262-4854-b701-930f85695c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:19.226126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:19.227324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:19.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:19.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:19.228380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:19.234900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 331/641] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:19.236714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7b17-7021-42d4-a8bd-b0ba571b707a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.237069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7b17-7021-42d4-a8bd-b0ba571b707a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.237501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7b17-7021-42d4-a8bd-b0ba571b707a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:19.263791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-039c7b17-7021-42d4-a8bd-b0ba571b707a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:19.265373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-039c7b17-7021-42d4-a8bd-b0ba571b707a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:19.267145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:19.268622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:19.351586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 332/642] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:19.352817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c7b17-7021-42d4-a8bd-b0ba571b707a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:19.405871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:19.412110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 311/643] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:19.414275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:19.491321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c7b17-7021-42d4-a8bd-b0ba571b707a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.502850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:19.532663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:19.534122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.534553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.534985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:19.542497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eac5f-e2e6-4623-a98e-16c738406195 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.550166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:19.556923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:19.558169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:19.559107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:19.561720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 333/644] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:19.562985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:19.585467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:19.586245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.586644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.587027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:19.587752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:19.588135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:19.589646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:19.607951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 312/645] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:19.607951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:19.694134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:19.709036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0a364-755f-4a4e-837f-6593c2f345b2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.752825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62f723-421d-4346-89fa-dce58e3b50bf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.764399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:19.765332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.765840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.766322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:19.786335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:19.789096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:19.798304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:19.798922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.799205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.799472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:19.800177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:19.800551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:19.802081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:19.813798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 313/646] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:19.814623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:19.824175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:19.825329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:19.826370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:19.826370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:19.826370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:19.827511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 334/647] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:19.828877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:19.855958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:19.949169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf905-e66b-482c-9fc4-213bcf7d2f92 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.957754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e27be3-5605-4cdc-be93-c9b6ccb77fca None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:19.968399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:19.994496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:19.995634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:19.996062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:19.996538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.002040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.003068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.003540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.004030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.006790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.008911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.017721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:20.023035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:20.024631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:20.033717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:20.034462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:20.036057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:20.037538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:20.039595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 314/648] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:20.039787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 335/649] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:20.039948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.040178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.153394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:20.161933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd464f0-386a-4e47-8e95-27d9294ba201 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.175732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31222ef-d479-4330-b958-8ecc00871dc7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.198208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.198881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.199115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.199444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.212105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.212516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.212761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.213023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.213310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.213971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:20.214271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.214382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.215647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.233061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 336/650] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:20.233061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:20.233333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:20.234282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:20.235200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:20.235200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:20.235200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:20.235200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 315/651] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:20.235200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.264368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:20.348094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be95c5-377a-4ad0-a209-0ae803961a12 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.348802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a59d8-de56-477f-a08f-d157507af67d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.394447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e849f-655e-467e-af57-a61a8690035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.394692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.395077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e849f-655e-467e-af57-a61a8690035b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.395245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.395436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e849f-655e-467e-af57-a61a8690035b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.395576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.395721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e849f-655e-467e-af57-a61a8690035b None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.395849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.396290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e849f-655e-467e-af57-a61a8690035b None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:20.396573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:20.396701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e849f-655e-467e-af57-a61a8690035b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.396822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.398051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e849f-655e-467e-af57-a61a8690035b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.398330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.409435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 337/652] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:20.411034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df908a17-204c-4401-abd1-4a070ed73efe None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.411363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df908a17-204c-4401-abd1-4a070ed73efe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.411634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df908a17-204c-4401-abd1-4a070ed73efe None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.413105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 338/653] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:20.413852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df908a17-204c-4401-abd1-4a070ed73efe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.419491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:20.421754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 316/654] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:20.423209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.522970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:20.535393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df908a17-204c-4401-abd1-4a070ed73efe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.536101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef733c-792e-4b64-9f01-a6c36defc850 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.576039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.576665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.576951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.577234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.577905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:20.578154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b068c-e7d3-41fe-98b8-c6c57df99ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.578284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.578784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b068c-e7d3-41fe-98b8-c6c57df99ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.579063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b068c-e7d3-41fe-98b8-c6c57df99ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.579333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b068c-e7d3-41fe-98b8-c6c57df99ab7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.579580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.586378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b068c-e7d3-41fe-98b8-c6c57df99ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.587960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b068c-e7d3-41fe-98b8-c6c57df99ab7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.592459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 317/655] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:20.593310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b60b068c-e7d3-41fe-98b8-c6c57df99ab7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:20.599625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:20.601133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 339/656] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae29a25-7281-4e1f-b103-2f67ffb37596 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.602837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae29a25-7281-4e1f-b103-2f67ffb37596 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.603857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae29a25-7281-4e1f-b103-2f67ffb37596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.621250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8ae29a25-7281-4e1f-b103-2f67ffb37596 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:20.622491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8ae29a25-7281-4e1f-b103-2f67ffb37596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:20.683617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:20.695588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 340/657] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:20.696528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae29a25-7281-4e1f-b103-2f67ffb37596 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.703128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984a91f-833f-4ee5-90f0-edeb6d7e561c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.739710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.740361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.740654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.740924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.754142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.755859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.768799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:20.777621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:20.778689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:20.779660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:20.779660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:20.779660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:20.781291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 318/658] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:20.782291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.840117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae29a25-7281-4e1f-b103-2f67ffb37596 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.900268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.901202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.901651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.902126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.903240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=2aba60973c5e4aa7b0b680acb24d0431) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:20.903741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.906236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:20.907892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868df1fb-19fc-48ad-a9b1-0dd79cfa76fb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:20.916176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 341/659] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/role_assignments?user.id=2aba60973c5e4aa7b0b680acb24d0431&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:20.917490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:20.973955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:20.975327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:20.975880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:20.976442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:20.986984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:20.988529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:21.002116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:21.003356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 319/660] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:21.004404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.025533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:21.081732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e6634-5036-4caf-8e3e-aeb1d27cca78 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.091045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:21.124650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.124650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.124940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.125067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:21.125818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:21.126152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:21.128137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:21.145932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 342/661] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:21.147422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46eb48-ee52-4076-a9b2-d23f82365c65 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.148507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.189551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.189551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.190038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.190739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:21.191636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:21.191636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:21.192595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:21.208540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 320/662] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:21.209385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.282707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091f16-c11f-4314-acbe-19c559288641 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.307125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:21.342141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.342895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.343306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.343706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:21.344690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187360b-eadc-4efa-b0c7-13d55bc34f95 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.360463 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:21.366455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:21.367567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:21.368571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:21.370780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 343/663] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:21.372269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.394227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:21.419855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.420640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.421022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.421457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:21.422736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:21.423228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:21.425642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:21.448281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 321/664] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:21.449177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.522166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54c559-02c1-4301-83cf-5c5c716944d5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.573952 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.575052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.575195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.575620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:21.576240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=bd45335f7a65416a8e2535c582f0624a) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:21.576558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:21.578061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:21.589758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 344/665] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/role_assignments?user.id=bd45335f7a65416a8e2535c582f0624a&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:21.589758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.608064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421c1e9-88ba-46ee-973a-b249d0a898a4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.663048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:21.665237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.665985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.666343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.667210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:21.678672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:21.680451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:21.698526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:21.700027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 12:52:21.701049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:21.701616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 322/666] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:21.705235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.783648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98667ca-8355-491b-ba15-333ebb7f5172 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.790570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:21.844302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.844956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.845238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.845507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:21.846244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:21.846788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:21.849475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:21.862445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 345/667] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:21.863295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:21.910872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894658-d0bc-46ce-a410-7119436dfb63 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:21.989759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:21.990806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:21.991247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:21.991729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.044423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580eb3-4018-494a-a1e2-3a081768193c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:22.047216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:22.105053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:22.106155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:22.106711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:22.107184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.108143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:22.108749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.110482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:22.138778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.140860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:22.145766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 346/668] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:22.147018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:22.254719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:22.266925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dad8f-d02d-404d-8251-fb6b1d5fb1fa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:22.272451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 323/669] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:22.273545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:22.306999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:22.308056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:22.308644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:22.309155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.322115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.323980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:22.340775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:22.341947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:22.342959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:22.342959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:22.342959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:22.344829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 347/670] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:22.346104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:22.363504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:22.410804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4a97-14fd-47c0-8fa9-cc84128b8041 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:22.451079 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:22.451716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:22.452073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:22.452380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.464028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:22.471263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fdccb-6319-438d-8bab-948eb5932efa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:22.513997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:22.515530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:22.516330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:22.517065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.531903 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.533777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:22.536943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.538383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:22.550200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:22.551250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:22.552257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:22.552257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:22.552257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:22.554471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 348/671] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:22.555787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:22.670287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:22.677767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 324/672] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:22.679181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:22.681215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7760882-c023-41bd-a5b3-80ca1411142f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:22.728261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:22.729002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:22.729359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:22.729528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:22.730046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.730890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:22.730890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.733116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:22.749843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 349/673] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:22.750920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:22.816092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606af9f-04cb-4aef-89d9-2bf2da7287bf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:22.870475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:22.871547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:22.872051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:22.872548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.873482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:22.874143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.876385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:22.891391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 325/674] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:22.892612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:22.898586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0a5ee-ca01-45f9-a8c6-1ab460e77ff0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:22.934151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:22.954457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:22.955510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:22.957085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:22.957611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:22.973649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:22.977128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:23.006055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:23.007101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:23.008029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:23.008029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:23.008029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:23.008029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 350/675] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:23.008898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.035820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684f8e-b7b6-4e56-a135-5a6907cc3860 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.046094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:23.084715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.086121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.086551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.087037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.098288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.100419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:23.116519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:23.117818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:23.119008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:23.120569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 326/676] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:23.121100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.197302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85c9ba-c49b-424b-8ee2-3ee529387ec4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.261337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ac6f-aa1e-4aba-b82c-d3354f6b38c5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.261816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.263302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.263825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.265972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.282902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.286391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:23.304881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:23.308006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 351/677] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:23.310315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.322891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.323526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.323802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.324175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.324829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:23.325227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.326642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.348333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:23.349001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 327/678] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:23.350992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.375417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:23.438535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ac823-ca2b-464b-86a8-f636afd41cac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.478647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.479509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.479866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.480259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.481121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:23.481505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.483275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.489755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64419eb-bc2d-4b34-b366-98677c5c12ff None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.504372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 352/679] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:23.505451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.535048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.535783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.536068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.536339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.537066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:23.537370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.538919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.559856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 328/680] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:23.560898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.630195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:23.639868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442019c7-cd12-4da7-90ec-73cc243799a8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.657183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:23.681857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.682561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.682792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc418a1e-9ec6-4d55-9796-c89693ea1b4b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.682894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.683169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.691914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.693897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:23.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:23.709371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 353/681] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:23.710544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.729855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d2f99-d548-45c2-b0bb-a1dd0cc66974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.731514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d2f99-d548-45c2-b0bb-a1dd0cc66974 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.731974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d2f99-d548-45c2-b0bb-a1dd0cc66974 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.732397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d2f99-d548-45c2-b0bb-a1dd0cc66974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.822477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d2f99-d548-45c2-b0bb-a1dd0cc66974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.823999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d2f99-d548-45c2-b0bb-a1dd0cc66974 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.865113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a390-8454-44cf-9226-7ed9d9a7b470 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:23.933639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b18b95-3b6d-42a1-8360-1f0a1d7b1130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:23.933907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:23.934763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b18b95-3b6d-42a1-8360-1f0a1d7b1130 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.935307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b18b95-3b6d-42a1-8360-1f0a1d7b1130 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.935724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b18b95-3b6d-42a1-8360-1f0a1d7b1130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.937130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b18b95-3b6d-42a1-8360-1f0a1d7b1130 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=42f946fc56244e179973d6f846eb7548) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:23.937712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b18b95-3b6d-42a1-8360-1f0a1d7b1130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:23.940229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b18b95-3b6d-42a1-8360-1f0a1d7b1130 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:23.957715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 354/682] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/role_assignments?user.id=42f946fc56244e179973d6f846eb7548&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:23.957715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96037a43-e5e0-4854-8819-84981f48abbc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.957715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96037a43-e5e0-4854-8819-84981f48abbc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.957715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96037a43-e5e0-4854-8819-84981f48abbc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.958150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 355/683] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:23.958926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96037a43-e5e0-4854-8819-84981f48abbc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:23.967132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 329/684] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:23.968564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860306d-bc74-43a1-9b4a-77a528c8aac4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:23.968859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860306d-bc74-43a1-9b4a-77a528c8aac4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:23.969139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860306d-bc74-43a1-9b4a-77a528c8aac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:23.992519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4860306d-bc74-43a1-9b4a-77a528c8aac4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:23.993924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4860306d-bc74-43a1-9b4a-77a528c8aac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:23.998234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:24.105940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 330/685] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:24.107474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860306d-bc74-43a1-9b4a-77a528c8aac4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:24.175353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96037a43-e5e0-4854-8819-84981f48abbc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:24.221560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:24.222412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.222804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.223082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.223792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:24.224097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:24.225581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:24.243564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 356/686] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:24.249530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:24.308651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860306d-bc74-43a1-9b4a-77a528c8aac4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:24.334985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:24.369799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:24.370526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.370817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.371099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.390422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:24.392390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:24.397238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:24.413313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:24.414491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:24.415453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:24.415453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:24.415453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:24.415453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 331/687] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:24.420803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:24.447518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e32d41-f1b3-499e-9837-a3968a147d3c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:24.489243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb2b01-75d1-4f72-a8d3-cd6905dafad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:24.489893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2b01-75d1-4f72-a8d3-cd6905dafad1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.490170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2b01-75d1-4f72-a8d3-cd6905dafad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.490432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2b01-75d1-4f72-a8d3-cd6905dafad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.491172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2b01-75d1-4f72-a8d3-cd6905dafad1 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=4302574365514eb1b25bd8e62736a4c5) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:24.491480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2b01-75d1-4f72-a8d3-cd6905dafad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:24.493489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2b01-75d1-4f72-a8d3-cd6905dafad1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:24.505241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 357/688] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:24 2022] GET /identity/v3/role_assignments?user.id=4302574365514eb1b25bd8e62736a4c5&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:24.505241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db5838-d411-4ed9-a017-0b9375c6b087 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.505241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db5838-d411-4ed9-a017-0b9375c6b087 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.505241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db5838-d411-4ed9-a017-0b9375c6b087 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.505241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 358/689] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:24.505241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db5838-d411-4ed9-a017-0b9375c6b087 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:24.553206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df386d9-357e-4054-86bf-478e3f9985b5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:24.604626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f4e4f-7ef8-4a83-8a6f-2f5a1f90bbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:24.605502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f4e4f-7ef8-4a83-8a6f-2f5a1f90bbad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.605856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f4e4f-7ef8-4a83-8a6f-2f5a1f90bbad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.606133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f4e4f-7ef8-4a83-8a6f-2f5a1f90bbad None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.606801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f4e4f-7ef8-4a83-8a6f-2f5a1f90bbad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:24.607109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f4e4f-7ef8-4a83-8a6f-2f5a1f90bbad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:24.608697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f4e4f-7ef8-4a83-8a6f-2f5a1f90bbad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:24.628863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 332/690] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:24.630567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593ff28-9876-4765-ab68-083f1403ca51 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.630858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593ff28-9876-4765-ab68-083f1403ca51 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.631137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593ff28-9876-4765-ab68-083f1403ca51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.632950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db5838-d411-4ed9-a017-0b9375c6b087 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:24.652881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3593ff28-9876-4765-ab68-083f1403ca51 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:24.654302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3593ff28-9876-4765-ab68-083f1403ca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:24.672271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:24.677423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:24.678591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.679078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.679537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.700213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:24.702645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:24.705044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:24.723032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:24.724174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:24.725105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:24.725105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:24.725105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:24.727471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 359/691] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:24.728832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:24.746869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 333/692] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:24.747899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593ff28-9876-4765-ab68-083f1403ca51 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:24.858326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73beb19-8627-4fb2-8e22-1753fcd7cc10 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:24.902416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:24.903498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.904114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.904666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.913543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:24.916285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:24.934166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:24.935190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:24.936128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:24.938517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 360/693] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:24.941339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593ff28-9876-4765-ab68-083f1403ca51 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:24.948346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:24.997260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:24.997941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:24.998219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:24.998486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:24.999147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:24.999455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:25.001905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:25.021217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 334/694] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:25.029334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:25.030036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:25.059296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:25.116099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072851ea-a039-4a6c-b6f3-ebdbf3598338 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:25.183804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:25.185105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:25.185860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:25.186409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:25.187793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:25.188399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:25.191562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:25.193230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc5e4b-bfdc-45af-ab00-1dedeae2fbca None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:25.216752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 361/695] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:25.218507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:25.239003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:25.240275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:25.240669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:25.241096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:25.242164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:25.244247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:25.278582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 335/696] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:52:25 2022] POST /identity/v3/users => generated 254 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:25.279943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:25.400583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826829d9-c5a5-49fb-a0dc-3a31f6509786 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:25.424177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:25.461729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:25.522521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:25.523168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:25.524113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:25.524514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:25.528802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1676442-184d-4684-9c57-1e2625ea253c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:25.551200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:25.556341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:25.557364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:25.558333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 362/697] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:25.566912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:25.591825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:25.592222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:25.592695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:25.593156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:25.594528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=30f70cdf08d049668d16007e4ca44654) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:25.595070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:25.597692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:25.609516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 336/698] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:25 2022] GET /identity/v3/role_assignments?user.id=30f70cdf08d049668d16007e4ca44654&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:25.733408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964ba84-f0ca-4866-80c0-6cd1101656b5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:25.777559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:25.778520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:25.778951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:25.779459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:25.780371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:25.780883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:25.782646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:25.795084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 363/699] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:25.801891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:25.968408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c249f-d7a2-48e9-b7c8-d94904395867 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:26.012782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:26.020050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:26.020709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:26.020989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:26.021264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:26.039036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:26.041904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:26.067164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:26.068448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:26.069475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:26.069475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:26.069475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:26.069689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 337/700] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:26.075446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:26.127129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:26.208266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fc1e6-fda4-40fb-8008-ed098fce8897 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:26.245484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:26.246629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:26.247071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:26.247513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:26.248477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:26.249039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:26.250686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:26.266371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 364/701] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:26.273917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:26.411460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fb6b7-6b9c-43b5-badc-616b1f44a0d8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:26.450476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:26.451124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:26.451405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:26.451680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:26.460774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:26.462395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:26.483798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:26.484921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:26.486113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 338/702] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:52:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:26.493113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:26.647533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:26.654720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694ff4d-60ce-4826-b2da-adc42bec9382 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:26.707745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665eb08-297a-4833-bb7a-9ba09fc66101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:26.708379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665eb08-297a-4833-bb7a-9ba09fc66101 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:26.708663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665eb08-297a-4833-bb7a-9ba09fc66101 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:26.708937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665eb08-297a-4833-bb7a-9ba09fc66101 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:26.709693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665eb08-297a-4833-bb7a-9ba09fc66101 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:26.710011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665eb08-297a-4833-bb7a-9ba09fc66101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:26.711531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665eb08-297a-4833-bb7a-9ba09fc66101 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:26.735739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 365/703] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:52:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:26.741622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:26.793243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:26.865057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4d64b-5399-4cc2-808a-5153629518e9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:26.865414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4d64b-5399-4cc2-808a-5153629518e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:26.865663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4d64b-5399-4cc2-808a-5153629518e9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:26.867707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 366/704] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:26.874185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fccc36-5ecb-4f9e-afe8-5870790d4a51 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:26.874576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fccc36-5ecb-4f9e-afe8-5870790d4a51 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:26.874931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fccc36-5ecb-4f9e-afe8-5870790d4a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:26.878452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6647-50b0-4b84-9331-f392e2000169 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:26.914618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-29fccc36-5ecb-4f9e-afe8-5870790d4a51 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:26.916379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-29fccc36-5ecb-4f9e-afe8-5870790d4a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:26.958280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:26.959903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:26.960610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:26.961764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:27.034536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 367/705] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:52:27.081344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:27.082845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:27.127856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:27.200441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 339/706] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:27.208175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fccc36-5ecb-4f9e-afe8-5870790d4a51 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:27.345651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:27.366930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fccc36-5ecb-4f9e-afe8-5870790d4a51 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:27.419854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:27.420510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:27.420792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:27.421171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:27.433925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:27.435442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:27.453324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:27.454408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:27.455344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:27.455344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:27.455344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:27.455344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 368/707] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:27.460276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:27.460482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:27.610296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ef01f-fded-4ced-892c-6cdf6c62745c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:27.648163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9844bf2-24a1-44c1-8596-79b20f14d2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:27.648878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9844bf2-24a1-44c1-8596-79b20f14d2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:27.649153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9844bf2-24a1-44c1-8596-79b20f14d2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:27.649430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9844bf2-24a1-44c1-8596-79b20f14d2f6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:27.650113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9844bf2-24a1-44c1-8596-79b20f14d2f6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:27.651488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9844bf2-24a1-44c1-8596-79b20f14d2f6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:27.658764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554853-0435-4fea-98f5-7a2288ad92a0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f9844bf2-24a1-44c1-8596-79b20f14d2f6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:27.662895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:27.663950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:27.665099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 340/708] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Sep 7 12:52:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:27.666022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3045fe-0f15-49c2-bc09-b40f5bdd5adf None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:27.666622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3045fe-0f15-49c2-bc09-b40f5bdd5adf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:27.666622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3045fe-0f15-49c2-bc09-b40f5bdd5adf None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:27.672154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 341/709] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:27.673026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3045fe-0f15-49c2-bc09-b40f5bdd5adf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:27.684452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:27.708043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e127e-368f-4185-b2f2-c2f80fa1d09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:27.708901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e127e-368f-4185-b2f2-c2f80fa1d09d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:27.709014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e127e-368f-4185-b2f2-c2f80fa1d09d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:27.709307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e127e-368f-4185-b2f2-c2f80fa1d09d None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:27.710030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e127e-368f-4185-b2f2-c2f80fa1d09d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:27.710344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e127e-368f-4185-b2f2-c2f80fa1d09d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:27.711818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e127e-368f-4185-b2f2-c2f80fa1d09d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:27.715162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:27.728055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 369/710] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:27.729489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2aefd-a921-4d97-ab0f-b0d8f741ac5a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:27.729807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2aefd-a921-4d97-ab0f-b0d8f741ac5a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:27.730084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2aefd-a921-4d97-ab0f-b0d8f741ac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:27.750643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-95c2aefd-a921-4d97-ab0f-b0d8f741ac5a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:27.752308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95c2aefd-a921-4d97-ab0f-b0d8f741ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:27.820745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3045fe-0f15-49c2-bc09-b40f5bdd5adf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:27.834235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 370/711] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:27.835788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2aefd-a921-4d97-ab0f-b0d8f741ac5a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:27.896048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:27.896776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:27.896969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:27.897253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:27.898061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:27.898372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:27.900381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:27.911420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 342/712] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 12:52:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:27.917527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:27.955610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2aefd-a921-4d97-ab0f-b0d8f741ac5a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:28.020360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:28.042575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:28.043627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.044125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.044601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.059545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:28.064273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:28.080731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:28.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:28.082816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:28.085100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 371/713] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:28.092018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:28.092314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:28.159258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57472f-6c90-472d-85f7-b926c4f0b069 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:28.210206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:28.211368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.211848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.212144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.213173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:28.213495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:28.215525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:28.223827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 343/714] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 12:52:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:28.224769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:28.283938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f118a99-d650-41f1-9b25-621e440eb41a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:28.388088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:28.389063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.389556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.390040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.390965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:28.391519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:28.394590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:28.430643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 372/715] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:28.436867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:28.480275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54dc9b-4624-4564-98e0-41cb5927f971 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:28.505878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:28.528583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe192f-b9a8-4472-936d-ef67652700bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:28.529738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe192f-b9a8-4472-936d-ef67652700bf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.530024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe192f-b9a8-4472-936d-ef67652700bf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.530298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe192f-b9a8-4472-936d-ef67652700bf None admin] PATH_INFO: `/v3/services/network` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.531170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe192f-b9a8-4472-936d-ef67652700bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:28.532586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe192f-b9a8-4472-936d-ef67652700bf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:28.535078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-febe192f-b9a8-4472-936d-ef67652700bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:28.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:28.545964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:28.547080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:28.554199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 344/716] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:28 2022] GET /identity/v3/services/network => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:28.556011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e476d-d4f8-428d-b56d-9f97dc80248a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.556454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e476d-d4f8-428d-b56d-9f97dc80248a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.556872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e476d-d4f8-428d-b56d-9f97dc80248a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.558925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 345/717] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:28.559898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e476d-d4f8-428d-b56d-9f97dc80248a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:28.606296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b97a5-5f2a-4c51-add6-c9ca8dbd34f1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:28.646342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297ceb0-554c-47f8-84f1-dbc561b81112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:28.647235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297ceb0-554c-47f8-84f1-dbc561b81112 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.647674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297ceb0-554c-47f8-84f1-dbc561b81112 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.648120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297ceb0-554c-47f8-84f1-dbc561b81112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.649170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297ceb0-554c-47f8-84f1-dbc561b81112 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=bd45335f7a65416a8e2535c582f0624a) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:28.649722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297ceb0-554c-47f8-84f1-dbc561b81112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:28.651494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297ceb0-554c-47f8-84f1-dbc561b81112 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:28.665921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 373/718] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:28 2022] GET /identity/v3/role_assignments?user.id=bd45335f7a65416a8e2535c582f0624a&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:28.667852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8594c98-8b25-4120-96b1-921393d54e7e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.668295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8594c98-8b25-4120-96b1-921393d54e7e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.668791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8594c98-8b25-4120-96b1-921393d54e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.711177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f8594c98-8b25-4120-96b1-921393d54e7e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:28.713487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f8594c98-8b25-4120-96b1-921393d54e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:28.749423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e476d-d4f8-428d-b56d-9f97dc80248a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:28.797944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73358a4e-bafc-4bb5-bd53-afaa25fb7ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:28.798897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73358a4e-bafc-4bb5-bd53-afaa25fb7ddc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.799451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73358a4e-bafc-4bb5-bd53-afaa25fb7ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.799930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73358a4e-bafc-4bb5-bd53-afaa25fb7ddc None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.800924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73358a4e-bafc-4bb5-bd53-afaa25fb7ddc None admin] RBAC: Adding query filter params (name=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:28.801428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73358a4e-bafc-4bb5-bd53-afaa25fb7ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:28.803095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73358a4e-bafc-4bb5-bd53-afaa25fb7ddc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:28.809894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 346/719] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:52:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:28.811759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38918a80-15fa-454e-aee7-550d3b322bc0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.812464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38918a80-15fa-454e-aee7-550d3b322bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.813084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38918a80-15fa-454e-aee7-550d3b322bc0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.815163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 347/720] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:28.816435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38918a80-15fa-454e-aee7-550d3b322bc0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:28.822738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 374/721] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:28.826497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137fa68-71fc-46a5-9977-4df94edb9ad0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:28.828277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137fa68-71fc-46a5-9977-4df94edb9ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:28.829755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137fa68-71fc-46a5-9977-4df94edb9ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:28.866221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b137fa68-71fc-46a5-9977-4df94edb9ad0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:28.869485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b137fa68-71fc-46a5-9977-4df94edb9ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:28.941856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:28.955707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:28.992535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38918a80-15fa-454e-aee7-550d3b322bc0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:29.040037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 375/722] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:29.047501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:29.048515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:29.049021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:29.049484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:29.050337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] RBAC: Adding query filter params (type=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:29.050841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:29.052488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:29.058764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 348/723] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:52:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:29.231921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137fa68-71fc-46a5-9977-4df94edb9ad0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:29.353499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:29.389046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137fa68-71fc-46a5-9977-4df94edb9ad0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:29.461846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:29.463078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:29.463555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:29.464029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:29.486535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:29.497039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:29.498158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:29.499054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:29.506543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 376/724] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:29.514653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:29.565479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8c218-88b9-4fd7-9d88-bec07d2aad14 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:29.580829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:29.654157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:29.670627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cc3a8-8631-4e00-ba45-cdf7f8105a0b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:29.696056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:29.696723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:29.697067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:29.697346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:29.721049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:29.722828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:29.723161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:29.723499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:29.724235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:29.724739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:29.728131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:29.730730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:29.736751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:29.737851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:29.738751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:29.742224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 349/725] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:29.750086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:29.751206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 377/726] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:29.757534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:29.906308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:29.922002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2e49f-b752-4a58-85a2-60f343337974 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:29.965645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:29.966672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:29.967129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:29.967573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:29.982420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:29.983918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:30.007367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:30.009577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:30.011379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:30.011379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:30.011379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:30.011379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 378/727] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:30.023860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:30.087047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49126dfe-c7b5-4b82-b9a3-465f9e8c7b77 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:30.182887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:30.184404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:30.185204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:30.186014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:30.187510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:30.188348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:30.191226 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:30.225325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 350/728] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:30.239541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:30.316377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa37e01-770e-496c-9959-9816e5c7ad5e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:30.343210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:30.408617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:30.409301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:30.409791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:30.409968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:30.410637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:30.410930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:30.412342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:30.448150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 379/729] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:30.456572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dad2b8-62b8-4066-86d8-8e7580735972 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:30.474827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:30.486838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:30.529622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:30.531257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:30.532043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:30.532796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:30.568789 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:30.574318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:30.617710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:30.619621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:30.621391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:30.621391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:30.621391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:30.621391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 351/730] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:30.621391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:30.741912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c50ac1e-afc2-4ae7-8011-9f88c2b31ddf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:30.780183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:30.780931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:30.781204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:30.781470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:30.789919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:30.792076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:30.808221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:30.810058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:30.811837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 380/731] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:52:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:30.823369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:30.845879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:30.850456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30077070-a89e-4045-9c43-acfc318d3fba None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:30.910548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:30.911486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:30.911923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:30.912346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:30.913142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:30.913588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:30.915545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:30.938141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 352/732] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:30.939573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:30.945950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:31.007496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0ae51-a57e-4074-9907-68bf1f4c52ec None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:31.078384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78655c26-4fe0-49e1-b933-ec13a26293b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:31.080116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655c26-4fe0-49e1-b933-ec13a26293b5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.080566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655c26-4fe0-49e1-b933-ec13a26293b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.080999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655c26-4fe0-49e1-b933-ec13a26293b5 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.082246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78655c26-4fe0-49e1-b933-ec13a26293b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:31.082921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78655c26-4fe0-49e1-b933-ec13a26293b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:31.084733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78655c26-4fe0-49e1-b933-ec13a26293b5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:31.111323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 381/733] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 12:52:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:31.113817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5e141-791d-4f93-90a6-1bdd1735e40b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.114474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5e141-791d-4f93-90a6-1bdd1735e40b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.115153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5e141-791d-4f93-90a6-1bdd1735e40b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.129855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 382/734] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:31 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:31.131585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5e141-791d-4f93-90a6-1bdd1735e40b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:31.183866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be581e0e-7c58-4147-bf55-5a3a56decd2f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:31.251711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e178b7c-d3a2-4069-811a-36a5da1d8952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:31.252523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178b7c-d3a2-4069-811a-36a5da1d8952 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.252901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178b7c-d3a2-4069-811a-36a5da1d8952 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.253264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178b7c-d3a2-4069-811a-36a5da1d8952 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.264105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e178b7c-d3a2-4069-811a-36a5da1d8952 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:31.266260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e178b7c-d3a2-4069-811a-36a5da1d8952 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9e178b7c-d3a2-4069-811a-36a5da1d8952 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:31.281808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:31.283194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:31.284522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:31.286728 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 353/735] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:31.288601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9f5ce-252f-4fa4-868d-6192f10fd8f1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.289708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9f5ce-252f-4fa4-868d-6192f10fd8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.290105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9f5ce-252f-4fa4-868d-6192f10fd8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.294992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5e141-791d-4f93-90a6-1bdd1735e40b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:31.315030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8db9f5ce-252f-4fa4-868d-6192f10fd8f1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:31.316744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8db9f5ce-252f-4fa4-868d-6192f10fd8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:31.325464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:31.340946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:31.343266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efdec3-c6e9-47a9-b87d-653c126c7b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:31.344653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efdec3-c6e9-47a9-b87d-653c126c7b6a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.345935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efdec3-c6e9-47a9-b87d-653c126c7b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.347299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efdec3-c6e9-47a9-b87d-653c126c7b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.425835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 354/736] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:31.427301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9f5ce-252f-4fa4-868d-6192f10fd8f1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:31.478695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efdec3-c6e9-47a9-b87d-653c126c7b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:31.482327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efdec3-c6e9-47a9-b87d-653c126c7b6a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:31.607343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9f5ce-252f-4fa4-868d-6192f10fd8f1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:31.629804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 383/737] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:31.631928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b030c-bb48-4a13-8695-93c665ebf3aa None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.634203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b030c-bb48-4a13-8695-93c665ebf3aa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.634570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b030c-bb48-4a13-8695-93c665ebf3aa None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.637240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 384/738] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:31.638273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b030c-bb48-4a13-8695-93c665ebf3aa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:31.667549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd800d4-5a1d-4bc9-ab29-93f76875b182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:31.668198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd800d4-5a1d-4bc9-ab29-93f76875b182 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.668480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd800d4-5a1d-4bc9-ab29-93f76875b182 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.668762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd800d4-5a1d-4bc9-ab29-93f76875b182 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.669543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd800d4-5a1d-4bc9-ab29-93f76875b182 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:31.669893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd800d4-5a1d-4bc9-ab29-93f76875b182 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:31.671500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd800d4-5a1d-4bc9-ab29-93f76875b182 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:31.675971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:31.700620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 355/739] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 12:52:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:31.702286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8d460-e483-462d-bf4e-5387cad11a86 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.702575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8d460-e483-462d-bf4e-5387cad11a86 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.702853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8d460-e483-462d-bf4e-5387cad11a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.723541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-29a8d460-e483-462d-bf4e-5387cad11a86 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:31.725249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-29a8d460-e483-462d-bf4e-5387cad11a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:31.728110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:31.781329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b030c-bb48-4a13-8695-93c665ebf3aa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:31.825546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 356/740] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:31.826221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:31.826758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8d460-e483-462d-bf4e-5387cad11a86 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:31.826919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:31.827406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:31.827406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:31.845871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:31.846763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:31.866620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:31.867787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:31.868699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:31.868699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:31.868699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:31.868699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 385/741] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:31.868699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:31.970064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8d460-e483-462d-bf4e-5387cad11a86 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:31.972637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:32.006166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe2776-4f3b-4661-b776-5299d9919482 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:32.017187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fd4f2-f4cb-4cf8-a484-8876bc84179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:32.018417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fd4f2-f4cb-4cf8-a484-8876bc84179f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.018847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fd4f2-f4cb-4cf8-a484-8876bc84179f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.019263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fd4f2-f4cb-4cf8-a484-8876bc84179f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.029358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:32.058088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e257559-cb98-4fee-b76a-ae13c57aaf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:32.058714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e257559-cb98-4fee-b76a-ae13c57aaf9d None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.058981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e257559-cb98-4fee-b76a-ae13c57aaf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.059249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e257559-cb98-4fee-b76a-ae13c57aaf9d None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.059907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e257559-cb98-4fee-b76a-ae13c57aaf9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:32.061479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e257559-cb98-4fee-b76a-ae13c57aaf9d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:32.089696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 386/742] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:52:31 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:32.089696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71605b-d6be-48c0-9931-d0a1d9de5e98 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.089696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71605b-d6be-48c0-9931-d0a1d9de5e98 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.089696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71605b-d6be-48c0-9931-d0a1d9de5e98 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.089696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 387/743] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:32.089696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71605b-d6be-48c0-9931-d0a1d9de5e98 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:32.145631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fd4f2-f4cb-4cf8-a484-8876bc84179f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:32.147388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fd4f2-f4cb-4cf8-a484-8876bc84179f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:32.248355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71605b-d6be-48c0-9931-d0a1d9de5e98 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:32.275257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 357/744] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:52:32.276724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b36bb1-4aad-4f8c-aeca-6ed4d5e9e337 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.277018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b36bb1-4aad-4f8c-aeca-6ed4d5e9e337 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.277292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b36bb1-4aad-4f8c-aeca-6ed4d5e9e337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.291913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:32.301528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:32.302451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.302871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.303296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.304098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:32.304403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:32.304556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e2b36bb1-4aad-4f8c-aeca-6ed4d5e9e337 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:32.306717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e2b36bb1-4aad-4f8c-aeca-6ed4d5e9e337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:32.306898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:32.320714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 388/745] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:32.321947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:32.348317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:32.411820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 358/746] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:52:32.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b36bb1-4aad-4f8c-aeca-6ed4d5e9e337 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:32.491208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a48631-f33d-4f34-8604-000a6d6de0d1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:32.550415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:32.551169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.551605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.551994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.569162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:32.569815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b36bb1-4aad-4f8c-aeca-6ed4d5e9e337 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:32.575811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:32.577878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 389/747] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:32.580063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:32.652890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:32.653357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:32.655481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.656148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.656777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.684201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:32.689001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:32.725444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:32.726667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:32.727654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:32.727654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:32.727654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:32.727997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 359/748] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:32.730180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:32.736963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6569807-4ba5-495c-ae0e-77a307a3d580 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:32.782240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:32.797089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:32.798066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:32.798609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:32.799138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:32.810843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:32.813410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:32.833649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:32.835436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:32.837863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 390/749] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:32.838823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:32.956912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3b02-84dd-4d7d-a0c3-82daa05d87a7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:32.999779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8aec6-0ad5-4502-990a-41786fa4e7b4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:33.010967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.011588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.011867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.012140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.012830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:33.013132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.014668 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.020637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:33.032777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 360/750] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:33.032777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:33.063962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.064613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.064899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.065224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.065911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:33.066209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.067616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.081509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 391/751] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:33.082070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:33.171324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:33.194066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdcb43-53ea-4c9b-9780-86f69894cc81 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:33.225374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45003d02-1e3f-419a-abaa-e21f77c9da07 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:33.237500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.238569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.238864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.239140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.239995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:33.240694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.242236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.261021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 361/752] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:33.261021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:33.269280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.270293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.270747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.271171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.272082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.274812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.301744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 392/753] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:52:33 2022] POST /identity/v3/services => generated 240 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:33.301744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:33.355637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:33.439507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c93e77-78d7-4399-a04e-8edfa65d5e6c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:33.447488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6931282-b404-4507-8d52-33e1dcb2449a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:33.488306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.489376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.490116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.490651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.491401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.492235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.492755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.493035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.501358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.503705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.505027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.506902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.510042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:33.516602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:33.517540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:33.518814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 393/754] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:33.519357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:33.544374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:33.546211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:33.547389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:33.547389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:33.547389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:33.552935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 362/755] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:33.555442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:33.631592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:33.655455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72850de-01cb-45b8-a286-faafaf361534 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:33.714520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20525d-f253-4411-95a8-120221d0f071 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:33.742224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.744252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.744867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.745468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.748067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1f9f3907a6d94b42b40343adca84bba2) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:33.748733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.751919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.765701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 394/756] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:33 2022] GET /identity/v3/role_assignments?user.id=1f9f3907a6d94b42b40343adca84bba2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:33.765701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:33.768288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:33.769014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:33.769343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:33.769620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:33.770370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:33.770681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:33.772248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:33.824041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 363/757] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:33.835157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:34.002920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:34.112712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541f035-ee68-428c-b3f1-8e387845bcd8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:34.117376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a318-915e-4b50-bf68-0a1f79beedea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:34.173436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:34.174152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:34.174432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:34.174692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:34.175373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:34.182111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:34.183908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:34.188014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:34.189463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:34.190290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:34.191044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:34.192612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=30f70cdf08d049668d16007e4ca44654) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:34.193433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:34.196297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:34.211156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 364/758] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:33 2022] GET /identity/v3/role_assignments?user.id=30f70cdf08d049668d16007e4ca44654&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:34.211751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 395/759] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:34.223377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:34.245928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:34.382038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb47b88-3b4e-4556-aeb2-3ca68671c04a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:34.435282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ed62d-daf8-4662-a350-9ecf01b0f43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:34.436506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ed62d-daf8-4662-a350-9ecf01b0f43e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:34.436803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ed62d-daf8-4662-a350-9ecf01b0f43e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:34.437260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ed62d-daf8-4662-a350-9ecf01b0f43e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:34.451472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ed62d-daf8-4662-a350-9ecf01b0f43e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:34.454218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ed62d-daf8-4662-a350-9ecf01b0f43e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-555ed62d-daf8-4662-a350-9ecf01b0f43e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:34.477969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:34.479901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:34.482187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 396/760] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:52:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:34.493654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:34.657059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2c0c5-709b-481a-89cd-08997353bf62 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:34.657254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2c0c5-709b-481a-89cd-08997353bf62 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:34.657991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2c0c5-709b-481a-89cd-08997353bf62 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:34.659446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 397/761] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:52:34.665070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f9264-3055-46fe-bc2d-4f863237f007 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:34.665353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957098e-5f85-4107-b2aa-49494778e2f7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:34.665641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957098e-5f85-4107-b2aa-49494778e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:34.665948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957098e-5f85-4107-b2aa-49494778e2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:34.688020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:34.703261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e957098e-5f85-4107-b2aa-49494778e2f7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:34.705390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e957098e-5f85-4107-b2aa-49494778e2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:34.738886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:34.739889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:34.740367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:34.740770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:34.741537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:34.741923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:34.743638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:34.769830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 365/762] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:52:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:34.771972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:34.848172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:34.858154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 398/763] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:52:35.000467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e4f73-ef8e-4b4c-906d-784698f7c646 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:35.059586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:35.060417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:35.060895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:35.061309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:35.090148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:35.092584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:35.118914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:35.120054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:35.121281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:35.121281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:35.121281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:35.121281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 366/764] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:35.126565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957098e-5f85-4107-b2aa-49494778e2f7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:35.253043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:35.320590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957098e-5f85-4107-b2aa-49494778e2f7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:35.335979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:35.392721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:35.393430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:35.393770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:35.394043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:35.395495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:35.395816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:35.397545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:35.417375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 399/765] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:35.427095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:35.471018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d0b14-1353-42d2-8e2a-799ba82ad507 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:35.512793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:35.513444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:35.513745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:35.513991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:35.514562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:35.516705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:35.540028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:35.541879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:35.543704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:35.550257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 367/766] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Sep 7 12:52:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:35.550628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:35.558275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:35.616727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5817f2-4d78-4f2b-a5e7-048ae9fb5218 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:35.656742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:35.657617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:35.657938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:35.658217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:35.667177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:35.669796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:35.692661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:35.695063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:35.696238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 400/767] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:35.703810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:35.850180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:35.873118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b951da-8c60-4bf0-aa12-710e81e02e1e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:35.897161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9ebe5-3968-4b7f-a766-ea2347c4c142 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:35.928237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:35.956156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:35.956928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:35.957352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:35.957851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:35.958625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:35.958935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:35.960416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:35.964591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:35.965316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:35.965596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:35.965953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:35.966872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:35.967170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:35.968706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:35.977261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 368/768] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 12:52:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:35.984029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:36.007685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 401/769] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:36.015918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:36.191312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a508d40-ae6e-4f63-92ce-94165aea43af None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:36.214333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4fde2-d403-475c-a11a-d36959cf1889 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:36.295264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:36.296266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:36.298516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:36.298977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:36.300054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:36.300557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:36.304697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:36.307756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28969f9e-3fce-4532-b8c7-1d78e9ac4f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:36.308840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969f9e-3fce-4532-b8c7-1d78e9ac4f97 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:36.309298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969f9e-3fce-4532-b8c7-1d78e9ac4f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:36.309740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969f9e-3fce-4532-b8c7-1d78e9ac4f97 None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/bd45335f7a65416a8e2535c582f0624a/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:36.315919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28969f9e-3fce-4532-b8c7-1d78e9ac4f97 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=bd45335f7a65416a8e2535c582f0624a, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:36.321451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28969f9e-3fce-4532-b8c7-1d78e9ac4f97 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:36.326436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 369/770] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 12:52:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:36.339284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:36.360362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 402/771] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:36 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/bd45335f7a65416a8e2535c582f0624a/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:52:36.396302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:36.441641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30753338-5b18-4699-bf20-890d05718faf None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:36.442572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30753338-5b18-4699-bf20-890d05718faf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:36.443274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30753338-5b18-4699-bf20-890d05718faf None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:36.453651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 403/772] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:52:36.456767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896570-abfa-4190-8618-ce201b3418af None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:36.457606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896570-abfa-4190-8618-ce201b3418af None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:36.458756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896570-abfa-4190-8618-ce201b3418af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:36.491461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-23896570-abfa-4190-8618-ce201b3418af None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:36.494176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-23896570-abfa-4190-8618-ce201b3418af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:36.569000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9db149-8d76-49ba-8fff-bd9adaebef03 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:36.630336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:36.656088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:36.657765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:36.660906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:36.665788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:36.667782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] RBAC: Adding query filter params (interface=public, service_id=78854fc58e864a7ab3e666f75a90b3fa, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:36.670386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:36.673201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:36.696378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 370/773] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:52:36 2022] GET /identity/v3/endpoints?service_id=78854fc58e864a7ab3e666f75a90b3fa&interface=public®ion_id=RegionOne => generated 194 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:36.706348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:36.708091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 404/774] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:36.836684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:36.866676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8c826-1028-466f-a374-06553cf2cc44 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:36.914582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee6fb5-066c-4e17-af4d-823a1a299ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:36.915737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee6fb5-066c-4e17-af4d-823a1a299ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:36.916270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee6fb5-066c-4e17-af4d-823a1a299ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:36.916720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee6fb5-066c-4e17-af4d-823a1a299ed5 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:36.917696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee6fb5-066c-4e17-af4d-823a1a299ed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:36.919439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee6fb5-066c-4e17-af4d-823a1a299ed5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:36.929000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 371/775] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:52:36 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:37.241556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896570-abfa-4190-8618-ce201b3418af None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:37.415228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d4de1-bf90-4d1a-81cc-d37edfb0170c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:37.415740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d4de1-bf90-4d1a-81cc-d37edfb0170c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:37.416152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d4de1-bf90-4d1a-81cc-d37edfb0170c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:37.422875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 372/776] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:37.429122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896570-abfa-4190-8618-ce201b3418af None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:37.439242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1abf84-eaed-4a0c-8969-4d824f20cae8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:37.439503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1abf84-eaed-4a0c-8969-4d824f20cae8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:37.439792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1abf84-eaed-4a0c-8969-4d824f20cae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:37.472253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ed1abf84-eaed-4a0c-8969-4d824f20cae8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:37.473273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed1abf84-eaed-4a0c-8969-4d824f20cae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:37.502988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:37.506888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:37.508461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:37.510436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:37.511503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:37.514050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:37.549386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:37.551282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:37.552756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:37.574247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 405/777] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:37 2022] GET /identity/v3/services/network => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:37.583703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:37.686839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 373/778] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:37.726759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:37.780470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9725-2219-4356-916c-cdf3d5c1ecb4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:37.818388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b678fee-0f6d-44f3-93d8-51b93fdc3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:37.819031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b678fee-0f6d-44f3-93d8-51b93fdc3e8f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:37.819311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b678fee-0f6d-44f3-93d8-51b93fdc3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:37.819582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b678fee-0f6d-44f3-93d8-51b93fdc3e8f None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:37.820426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b678fee-0f6d-44f3-93d8-51b93fdc3e8f None admin] RBAC: Adding query filter params (name=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:37.820732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b678fee-0f6d-44f3-93d8-51b93fdc3e8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:37.822197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b678fee-0f6d-44f3-93d8-51b93fdc3e8f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:37.832419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 406/779] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:52:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:37.833884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1abf84-eaed-4a0c-8969-4d824f20cae8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:37.924220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:37.941341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6037370-fe5f-409e-9f97-44fb6901764b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:37.941708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6037370-fe5f-409e-9f97-44fb6901764b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:37.941937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6037370-fe5f-409e-9f97-44fb6901764b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:37.943422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 407/780] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:52:37.949976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e422940-5798-42c5-b094-da776a0d6411 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:37.950493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e422940-5798-42c5-b094-da776a0d6411 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:37.951024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e422940-5798-42c5-b094-da776a0d6411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:37.953926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1abf84-eaed-4a0c-8969-4d824f20cae8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:37.992401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:37.993720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:37.994068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:37.994345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:37.994995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] RBAC: Adding query filter params (type=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:37.995303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:37.996706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:38.002938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 374/781] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:52:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:38.011312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:38.022593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4e422940-5798-42c5-b094-da776a0d6411 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:38.027372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4e422940-5798-42c5-b094-da776a0d6411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:38.150059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 408/782] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:38.151010 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e422940-5798-42c5-b094-da776a0d6411 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:38.180449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5968-fc89-4319-882b-589c7a078255 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:38.221833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:38.222280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:38.222869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:38.223266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:38.223561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:38.224493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] RBAC: Adding query filter params (interface=public, service_id=8ecbf6b55937489eaa4806f8d6fa4552, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:38.224813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:38.226778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:38.233303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 375/783] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:52:38 2022] GET /identity/v3/endpoints?service_id=8ecbf6b55937489eaa4806f8d6fa4552&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:38.241297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:38.258172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:38.427234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1915d91-7a44-4aae-8e3f-6d3f0e2de063 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:38.432790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e422940-5798-42c5-b094-da776a0d6411 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:38.465435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:38.466113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:38.466400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:38.467270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:38.468000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:38.469571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:38.475766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 376/784] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:52:38 2022] GET /identity/v3/services => generated 772 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:38.493717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:38.494641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:38.495018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:38.495389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:38.508782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:38.514409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:38.515453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:38.516313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 409/785] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:38.522572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:38.610101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:38.645465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6da7e-7333-49a1-ae1b-bcae60ef42ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:38.685911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:38.686550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:38.686833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:38.687184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:38.687878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:38.688185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:38.689600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:38.700751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:38.704771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 377/786] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:38.706643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:38.836631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8b64-4b33-4dae-a3f8-e8c087d803cd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:38.874149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:38.889388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd0b8d-e11d-4b72-a858-905d212a2eac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:38.891991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:38.900246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:38.901324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:38.902331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:38.902863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:38.917562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:38.920333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:38.941113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:38.942188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:38.943125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:38.943125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:38.943125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:38.946535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 378/787] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:38.960112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:38.968765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:38.969743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:38.970083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:38.970376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:38.989506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:38.996892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:38.998057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:38.999094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:39.001042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 410/788] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:39.011565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:39.227990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e00a0e-29a7-4b96-a921-13e802be70c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:39.341264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:39.342353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:39.342841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:39.343481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:39.344617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:39.345146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:39.347763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:39.352782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca41e-3ada-4b21-a207-d729ba38954c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:39.366875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 411/789] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:39.376941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:39.437708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:39.497698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:39.497698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:39.497698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:39.498065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:39.499309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:39.505709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:39.505709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:39.561964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 379/790] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 602 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:39.570695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:39.730180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:39.758164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5fa9a-7c10-4556-a896-e62ba7624c79 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:39.836232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe595-e4d1-438d-9b46-eae1bd27f186 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:39.842223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268472-58a6-4ac0-8c4a-54235426dd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:39.842888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268472-58a6-4ac0-8c4a-54235426dd20 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:39.843164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268472-58a6-4ac0-8c4a-54235426dd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:39.843435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268472-58a6-4ac0-8c4a-54235426dd20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:39.875815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268472-58a6-4ac0-8c4a-54235426dd20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:39.882389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268472-58a6-4ac0-8c4a-54235426dd20 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:39.914127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:39.916303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:39.917132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:39.917966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c4268472-58a6-4ac0-8c4a-54235426dd20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:39.923770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:39.925077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:39.926356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:39.926356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:39.926356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:39.928891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 412/791] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 553 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:39.930697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130ea8a-02c3-43f8-967c-b844759cb8c1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:39.931267 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130ea8a-02c3-43f8-967c-b844759cb8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:39.931793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130ea8a-02c3-43f8-967c-b844759cb8c1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:39.938049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 413/792] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:39.940221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cd63-895d-47c8-b885-4a64ebbc92fa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:39.940770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cd63-895d-47c8-b885-4a64ebbc92fa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:39.941238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cd63-895d-47c8-b885-4a64ebbc92fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:39.942873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:39.947076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:39.966022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-09e6cd63-895d-47c8-b885-4a64ebbc92fa None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:39.967601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-09e6cd63-895d-47c8-b885-4a64ebbc92fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:39.974298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:39.975350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:39.976384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:39.977393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 380/793] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:52:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:39.979255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.043490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:40.057511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 414/794] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:52:40.066779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6cd63-895d-47c8-b885-4a64ebbc92fa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.144185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f9dab-6b70-4944-84e8-1c314ed723ed None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:40.147110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:40.190132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:40.191170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.191747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.192227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.193227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:40.193801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:40.196019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:40.212126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 381/795] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:40.218535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.242677 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6cd63-895d-47c8-b885-4a64ebbc92fa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:40.350008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c80858-e6f3-4d28-a532-21e6ee2ab01f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:40.393402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d086ddc-3aef-40f6-a036-bd63261fc8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:40.394077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d086ddc-3aef-40f6-a036-bd63261fc8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.394360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d086ddc-3aef-40f6-a036-bd63261fc8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.394634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d086ddc-3aef-40f6-a036-bd63261fc8c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.405696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d086ddc-3aef-40f6-a036-bd63261fc8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:40.406046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d086ddc-3aef-40f6-a036-bd63261fc8c8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:40.408425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a69c68-a3c8-484f-8d2e-8c4e79f6eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:40.409124 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69c68-a3c8-484f-8d2e-8c4e79f6eb33 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.409401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69c68-a3c8-484f-8d2e-8c4e79f6eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.409725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69c68-a3c8-484f-8d2e-8c4e79f6eb33 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.410463 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a69c68-a3c8-484f-8d2e-8c4e79f6eb33 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:40.410764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a69c68-a3c8-484f-8d2e-8c4e79f6eb33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:40.412207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a69c68-a3c8-484f-8d2e-8c4e79f6eb33 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9d086ddc-3aef-40f6-a036-bd63261fc8c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:40.420200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:40.421223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:40.422246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:40.426375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 382/796] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:40.427831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0cd07-cdb7-4752-b1f4-4c7167bd42c6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.428121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0cd07-cdb7-4752-b1f4-4c7167bd42c6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.428398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0cd07-cdb7-4752-b1f4-4c7167bd42c6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.430600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 383/797] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:40.433851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 415/798] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 12:52:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:40.435844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0cd07-cdb7-4752-b1f4-4c7167bd42c6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.436981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84903c5f-eca9-4101-9903-cc39423babd6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.437386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84903c5f-eca9-4101-9903-cc39423babd6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.437781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84903c5f-eca9-4101-9903-cc39423babd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.457740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-84903c5f-eca9-4101-9903-cc39423babd6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:40.459385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84903c5f-eca9-4101-9903-cc39423babd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:40.467767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:40.515995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:40.542705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 416/799] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:40.543733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84903c5f-eca9-4101-9903-cc39423babd6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.565161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0cd07-cdb7-4752-b1f4-4c7167bd42c6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:40.614971 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:40.616182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.617377 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.618649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.619785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:40.620305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:40.622404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:40.652008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 384/800] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 12:52:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:40.652008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.672542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84903c5f-eca9-4101-9903-cc39423babd6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:40.726227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:40.728049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.728805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.729561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.750056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:40.754668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:40.758991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:40.772151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956b1a4-417a-416e-8c23-48fcb67d8ac7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:40.792794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:40.794708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:40.796491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:40.796491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:40.796491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:40.800783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 417/801] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:40.802762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.827612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:40.828219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:40.828701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:40.829164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:40.852368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:40.862832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:40.864771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:40.867027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 385/802] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:40.868480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:40.879023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:41.012495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0765ca-81cf-42bf-8be9-1c80f81c6880 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.037089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676870f-9102-43ad-b942-f57ea912f7f6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.079646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.080295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.080651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.080963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.085878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.086920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.087399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.087884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.089057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:41.089605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.092307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.094106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.096071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.120466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 386/803] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:41.120466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:41.121743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:41.122704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:41.123845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:41.123845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:41.123845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:41.125976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:41.131718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 418/804] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:41.134091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:41.276357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:41.294490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5255c-a2a1-40a5-81f9-18dca37226be None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.314991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af132032-ce3b-44d5-a6cc-5f0aebe22b70 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.354081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.355398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.357004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.357491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.358777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.362204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.380682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:41.381205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:41.383354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 12:52:41.390933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:41.391711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.391711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.391711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.391929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 387/805] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:41.392000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:41.392000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.392132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:41.392204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.404688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 419/806] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:41.406485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:41.512295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:41.580278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0520-9512-4924-bd9a-c9cbb1fc8988 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.606404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f8b19-1f40-4a27-bddd-9a8367e25713 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.631730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.632727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.633268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.633776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.642140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.643727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:41.658287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:41.667933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 388/807] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:41.670103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:41.688639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.689879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.690456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.690948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.692019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:41.692332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.693824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.711823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 420/808] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:41.713150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:41.727679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:41.816416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379e7ae-a9c1-498b-a2bb-0360d98fb8b3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.852527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:41.867297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.868271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.868694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.869141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.870128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:41.870580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.872877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.873789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac7c3-eb65-4e45-91d3-0109aa5bce90 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:41.880872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 389/809] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:41.882546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:41.923141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231d41c-1497-4ba1-a86c-8c023d7d9b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:41.923796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231d41c-1497-4ba1-a86c-8c023d7d9b84 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.924076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231d41c-1497-4ba1-a86c-8c023d7d9b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.924343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231d41c-1497-4ba1-a86c-8c023d7d9b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.937857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231d41c-1497-4ba1-a86c-8c023d7d9b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:41.939442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231d41c-1497-4ba1-a86c-8c023d7d9b84 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3231d41c-1497-4ba1-a86c-8c023d7d9b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:41.962566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:41.964438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:41.966241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:41.966241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:41.966241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:41.966241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 421/810] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:41.966241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36d268-dd42-4e57-be62-09db1c63200a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:41.966241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36d268-dd42-4e57-be62-09db1c63200a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:41.966945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36d268-dd42-4e57-be62-09db1c63200a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:41.968401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 422/811] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:41.969841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36d268-dd42-4e57-be62-09db1c63200a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:42.046028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e78616-6f9e-4607-ba4a-5929ebf424cc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.064500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:42.099411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.100443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.101014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.101510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.105244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:42.105244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.106218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:42.126968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36d268-dd42-4e57-be62-09db1c63200a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.138625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 390/812] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:42.138625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:42.177152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.178160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.178575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.179106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.191058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.193478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:42.206644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:42.207424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:42.217312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 423/813] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:42.219023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:42.273627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37fe77-dbe7-4e5b-8adc-24c139a10baf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.326063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ae69f-27ac-487e-a349-3fdc20dfe873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.326972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae69f-27ac-487e-a349-3fdc20dfe873 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.327419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae69f-27ac-487e-a349-3fdc20dfe873 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.327849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae69f-27ac-487e-a349-3fdc20dfe873 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.328886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ae69f-27ac-487e-a349-3fdc20dfe873 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:42.329375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ae69f-27ac-487e-a349-3fdc20dfe873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.331784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ae69f-27ac-487e-a349-3fdc20dfe873 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:42.339118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 391/814] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 12:52:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:42.340567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1d65-6f70-4243-a007-391c66d1ba3c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.340923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1d65-6f70-4243-a007-391c66d1ba3c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.341263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1d65-6f70-4243-a007-391c66d1ba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.352104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720b623-8a2e-48e0-bfa5-fad94e81acd3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.366192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-630c1d65-6f70-4243-a007-391c66d1ba3c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:42.368448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-630c1d65-6f70-4243-a007-391c66d1ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:42.384148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:42.416307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.416853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.417134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.417409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.418121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:42.418450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.420071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:42.435824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 424/815] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:42.436700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:42.480339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 392/816] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:42.485982 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1d65-6f70-4243-a007-391c66d1ba3c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:42.548762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:42.602886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0cac2-7d95-4e75-a359-3db5dea8b3a8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.642390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1d65-6f70-4243-a007-391c66d1ba3c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.663482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.665074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.665074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.665404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1f9f3907a6d94b42b40343adca84bba2/roles/bca79ab169d64049a6983bc348d0c161` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.685817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=1f9f3907a6d94b42b40343adca84bba2, role_id=bca79ab169d64049a6983bc348d0c161)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.687811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:42.708090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.708655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.708949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.709189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.709965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:42.711088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.712468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:42.720603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 425/817] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:42 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1f9f3907a6d94b42b40343adca84bba2/roles/bca79ab169d64049a6983bc348d0c161 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:52:42.732741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 393/818] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:42.734906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:42.735115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:42.801587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:42.937393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9957f38-1d43-48d7-8ede-6e989bafcc46 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.945911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26de898-2ac6-47c0-8c73-766bb0164c01 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:42.947923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:42.986351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.987008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.987340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.987617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.992975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:42.993557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:42.993955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:42.994231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:42.994930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.996343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:42.997068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:42.998648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:43.013461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:43.015156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 394/819] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:52:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:43.018379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:43.031568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 426/820] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:52:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:43.032439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:43.180246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f83f52-cdef-4920-a563-5c935e7e206d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:43.219299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:43.256721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071970da-46a9-407d-8aa5-36a665bc9143 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:43.258913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:43.259705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:43.260085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:43.260419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/30f70cdf08d049668d16007e4ca44654/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:43.263520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=30f70cdf08d049668d16007e4ca44654, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:43.264299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:43.290061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 395/821] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:52:43 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/30f70cdf08d049668d16007e4ca44654/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:52:43.291213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:43.342120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:43.343167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:43.343536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:43.343867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] PATH_INFO: `/v3/services/network` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:43.344564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:43.348223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:43.357022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:43.366727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:43.368331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 12:52:43.369454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:43.373385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 427/822] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:43 2022] GET /identity/v3/services/network => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:43.385742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:43.607154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a4cf1-7ece-4d85-908e-8bd1652468ed None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:43.671608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:43.672683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:43.672683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:43.672927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:43.673472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] RBAC: Adding query filter params (name=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:43.673803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:43.681710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:43.682428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7aedca-d21d-4e6d-a055-a2646116ee00 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:43.693627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 428/823] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:52:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:43.704297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:43.730104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:43.786287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:43.786943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:43.787272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:43.787551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:43.788719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:43.789092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:43.790747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:43.843049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 396/824] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:52:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 552 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:43.849142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:43.948971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45305-5b95-41a0-9220-df206a9bb29c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:44.040846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:44.047514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:44.048162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:44.048488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:44.048818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:44.049490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] RBAC: Adding query filter params (type=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:44.049828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:44.051254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:44.073725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 429/825] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:52:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:44.083256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f87bc5-b7e3-49c7-a77d-325ac889b5e3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:44.093904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:44.146673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:44.147365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:44.147675 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:44.148027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:44.182118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:44.218007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0931b-bf32-49c8-9ffd-1c367f48145d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:44.247917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:44.249507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:44.308251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:44.309289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:44.309762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:44.310213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:44.311294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:44.312918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:44.388006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 430/826] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:52:44 2022] POST /identity/v3/endpoints => generated 337 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:52:44.451039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:44.484886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 397/827] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 12:52:44.490287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:44.676743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf7da-8b72-4c28-8eea-1dab4e27b269 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:44.706895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:44.724332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:44.724988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:44.725267 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:44.725533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:44.738293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:44.739864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:44.756920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:44.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:44.760649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:44.760649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:44.760649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:44.760649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 431/828] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:44.766879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:44.907371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132f275-6c74-4630-a5db-18a7a58232e9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:44.956570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:44.957199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:44.957479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:44.957858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:44.958513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:44.958820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:44.960447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:44.974643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 398/829] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:44.981906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:45.113288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264711-2042-4e95-9842-99e9f339f887 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:45.149626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:45.150773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:45.151269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:45.151730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:45.158906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:45.160600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:45.172771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:45.173745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:45.174673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:45.177043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 432/830] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:45.182628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:45.263509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:45.358780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969b8d3-cc3d-4c2e-b630-b314fedba36c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:45.430162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c3c0d-1421-4c97-8d4a-e4a120d8ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:45.430798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3c0d-1421-4c97-8d4a-e4a120d8ef9e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:45.431081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3c0d-1421-4c97-8d4a-e4a120d8ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:45.431359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3c0d-1421-4c97-8d4a-e4a120d8ef9e None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:45.432067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c3c0d-1421-4c97-8d4a-e4a120d8ef9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:45.432374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c3c0d-1421-4c97-8d4a-e4a120d8ef9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:45.433900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c3c0d-1421-4c97-8d4a-e4a120d8ef9e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:45.449187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 399/831] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:45.454792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:45.495494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:45.581382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df244a-fa92-40a9-a6e1-cc72a6ce4129 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:45.581653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df244a-fa92-40a9-a6e1-cc72a6ce4129 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:45.582073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df244a-fa92-40a9-a6e1-cc72a6ce4129 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:45.583780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 400/832] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:52:45.589511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83548b27-790d-4f8f-90c1-67cd06063bf1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:45.589867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83548b27-790d-4f8f-90c1-67cd06063bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:45.590343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83548b27-790d-4f8f-90c1-67cd06063bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:45.610224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-83548b27-790d-4f8f-90c1-67cd06063bf1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:45.611478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211cf06-a9f5-445e-85e4-8ef5fe0ddf58 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:45.611632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-83548b27-790d-4f8f-90c1-67cd06063bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:45.682356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e544b-7a38-4eca-b786-cb6e118b3c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:45.684503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e544b-7a38-4eca-b786-cb6e118b3c95 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:45.685357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e544b-7a38-4eca-b786-cb6e118b3c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:45.686145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e544b-7a38-4eca-b786-cb6e118b3c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:45.688077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e544b-7a38-4eca-b786-cb6e118b3c95 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=bd45335f7a65416a8e2535c582f0624a) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:45.688924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e544b-7a38-4eca-b786-cb6e118b3c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:45.692197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e544b-7a38-4eca-b786-cb6e118b3c95 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:45.701884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 401/833] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:52:45.710742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 433/834] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:45 2022] GET /identity/v3/role_assignments?user.id=bd45335f7a65416a8e2535c582f0624a&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:45.900414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533418cd-b44a-470f-bd91-1de318376daa None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:45.900843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533418cd-b44a-470f-bd91-1de318376daa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:45.901020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533418cd-b44a-470f-bd91-1de318376daa None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:45.902434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 402/835] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:45.908596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6496b5-1b2b-4993-b722-11bd3e33771d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:45.908982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6496b5-1b2b-4993-b722-11bd3e33771d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:45.909138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6496b5-1b2b-4993-b722-11bd3e33771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:45.928988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ed6496b5-1b2b-4993-b722-11bd3e33771d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:45.930523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ed6496b5-1b2b-4993-b722-11bd3e33771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:46.009449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 434/836] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:46.225085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533418cd-b44a-470f-bd91-1de318376daa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:46.262636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:46.296369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6496b5-1b2b-4993-b722-11bd3e33771d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:46.330993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:46.343022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533418cd-b44a-470f-bd91-1de318376daa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:46.384305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564b32f-8ab9-494f-9897-45d2044d2576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:46.384983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b32f-8ab9-494f-9897-45d2044d2576 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.385266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b32f-8ab9-494f-9897-45d2044d2576 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:46.385540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b32f-8ab9-494f-9897-45d2044d2576 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:46.401930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564b32f-8ab9-494f-9897-45d2044d2576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1564b32f-8ab9-494f-9897-45d2044d2576 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:46.410983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:46.412034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:46.412907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 403/837] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:52:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:46.416153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219a07d-951c-4f55-bddd-150f46d303f7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.416655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219a07d-951c-4f55-bddd-150f46d303f7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:46.416967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219a07d-951c-4f55-bddd-150f46d303f7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:46.419496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 404/838] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:46.420900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219a07d-951c-4f55-bddd-150f46d303f7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:46.444528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6496b5-1b2b-4993-b722-11bd3e33771d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:46.501427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597b057-0a40-4a35-be69-57c4f81d2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:46.502361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597b057-0a40-4a35-be69-57c4f81d2c9b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.502852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597b057-0a40-4a35-be69-57c4f81d2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:46.503315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597b057-0a40-4a35-be69-57c4f81d2c9b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:46.504067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597b057-0a40-4a35-be69-57c4f81d2c9b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:46.506152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597b057-0a40-4a35-be69-57c4f81d2c9b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5597b057-0a40-4a35-be69-57c4f81d2c9b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:46.517447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:46.522385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:46.523275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 435/839] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:46.524099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b154fc-aec6-459a-bdf4-a7866547c103 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.524099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b154fc-aec6-459a-bdf4-a7866547c103 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:46.524099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b154fc-aec6-459a-bdf4-a7866547c103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:46.540683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:46.547438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-68b154fc-aec6-459a-bdf4-a7866547c103 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:46.548845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-68b154fc-aec6-459a-bdf4-a7866547c103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:46.554307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219a07d-951c-4f55-bddd-150f46d303f7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:46.600217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:46.600920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.601244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:46.601530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:46.602229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:46.602490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:46.603911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:46.614426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 405/840] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:52:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:46.620194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:46.637174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 436/841] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:46.638071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b154fc-aec6-459a-bdf4-a7866547c103 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:46.679209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:46.751906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a417b0-5d8f-4cb7-aad7-35a23803f695 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:46.774846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b154fc-aec6-459a-bdf4-a7866547c103 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:46.791435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:46.792141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.792503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:46.792783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:46.793514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:46.794323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:46.795581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:46.801708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 406/842] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:46.810646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:46.817938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:46.818589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.818912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:46.819192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:46.831559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:46.833167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:46.838916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:46.852665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:46.853944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:46.854883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:46.854883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:46.854883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:46.854883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 437/843] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:46.859789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:46.940901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c72a3d-a19b-4c3f-9273-a1769875705b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:46.975734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:46.997648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:46.998977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:46.999727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.000159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.001976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:47.002441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:47.003796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca94e2-2f9e-4976-8339-88e9491a5bbe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:47.004904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:47.012848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 407/844] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:47.053244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:47.053988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:47.054279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.054551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.054803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:47.055224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:47.055532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:47.057186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:47.073209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 438/845] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:47.079115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:47.191609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:47.212149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63e1e2-f3a5-40fe-96c8-93bb71de32d4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:47.225441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1ab52-5ef5-42e9-9452-ffc24eec189b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:47.253287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:47.253988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:47.254270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.254545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.257774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:47.267028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:47.269196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:47.282910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:47.285044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:47.287273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 439/846] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:52:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:47.290362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:47.304137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:47.305579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:47.306313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.306984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.327721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:47.339191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:47.341357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:52:47.343429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:47.347634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 408/847] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:47.358588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:47.412552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a5968-e491-4c96-a205-79a38f99557e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:47.453288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:47.454277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:47.454559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.454834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.455845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:47.456160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:47.457739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:47.483531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 440/848] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 12:52:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:47.484696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:47.547015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a8e7-03e0-4f28-ae00-c3589500dc78 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:47.552947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:47.612014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:47.612554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:47.612844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.613106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.613833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:47.614141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:47.615575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:47.633764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 409/849] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:47.635443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:47.642190 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:47.666186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab89823-858f-4cf0-b55f-7bb91b369ed1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:47.745891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:47.746850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:47.747281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.747690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.823224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c35da-3e97-45af-86f7-2a9e2ed1d317 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:47.876334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:47.876538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:47.877907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:47.878493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:47.879045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:47.879153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:47.895029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:47.896990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:47.918683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:47.919855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:47.920898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:47.920898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:47.920898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:47.929012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 410/850] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:47.931191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:47.977286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.018886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.031820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 441/851] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 12:52:48.037377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:48.077429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053153f-fc76-4a7d-8f8f-c31a7a7e6c3c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.119320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.120344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.120825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.121317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.122196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:48.122726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.124314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.144146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 411/852] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:48.145445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:48.178809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba650b-88a8-4824-a504-ff0336ff46b0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.239342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.240067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.240372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.240645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.259000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.260883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.278498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3771c-c81f-4c5f-a7ae-d6e4b0e3b7ea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:48.282232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:48.283198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:48.284128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:48.284128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:48.284128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:48.286215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 442/853] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:48.292063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:48.300944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.304919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.325052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.325773 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.326063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.326341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.336751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.338256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:48.354376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:48.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 12:52:48.356463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:48.360424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 412/854] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:48.361720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:48.448594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed304-b239-4492-a80f-b5df22259bd6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.491598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361ef3e-35ee-4798-8038-d4f1999c1049 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.497957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.498726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.499128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.499487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.500614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:48.501828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.504109 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.525858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 443/855] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:48.528416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:48.539495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.540115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.540393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.540735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.541464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:48.541817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.543276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.563183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 413/856] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:48.568961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:48.610334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.630753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.669808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6979b-2571-46b6-99f7-b142c02bc7ae None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.683356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2442817-532c-429b-b880-1aff47c0c504 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.708555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05636f5-8ca9-4754-b49d-ba694c86938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.709178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05636f5-8ca9-4754-b49d-ba694c86938f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.709462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05636f5-8ca9-4754-b49d-ba694c86938f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.709765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05636f5-8ca9-4754-b49d-ba694c86938f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.717103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05636f5-8ca9-4754-b49d-ba694c86938f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.718608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05636f5-8ca9-4754-b49d-ba694c86938f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.720036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.720809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.721388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.721679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d05636f5-8ca9-4754-b49d-ba694c86938f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:48.730234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:48.731227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:48.732184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 444/857] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:48.732711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0768ae3-1247-45d0-af1c-805ed6d1414e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.733742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0768ae3-1247-45d0-af1c-805ed6d1414e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.733742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0768ae3-1247-45d0-af1c-805ed6d1414e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.734563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 445/858] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:48.739070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0768ae3-1247-45d0-af1c-805ed6d1414e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:48.804426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.806800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.848631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0768ae3-1247-45d0-af1c-805ed6d1414e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:48.855216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.880346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:48.890426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a201af-fd81-4082-a78d-7edb6264f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:48.891067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a201af-fd81-4082-a78d-7edb6264f118 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.891350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a201af-fd81-4082-a78d-7edb6264f118 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.891625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a201af-fd81-4082-a78d-7edb6264f118 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.892357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a201af-fd81-4082-a78d-7edb6264f118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:48.892668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a201af-fd81-4082-a78d-7edb6264f118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:48.894216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a201af-fd81-4082-a78d-7edb6264f118 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:48.913708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 446/859] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:48.913708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800570d2-984a-4df2-91f0-51324af09eff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:48.913708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800570d2-984a-4df2-91f0-51324af09eff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:48.913708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800570d2-984a-4df2-91f0-51324af09eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:48.932204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-800570d2-984a-4df2-91f0-51324af09eff None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:48.933423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-800570d2-984a-4df2-91f0-51324af09eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:48.942171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 414/860] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 12:52:48.943216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.024124 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 447/861] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:49.024916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800570d2-984a-4df2-91f0-51324af09eff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.083169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748d46e-f3ad-4796-9e87-d98f73bab70e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:49.128138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:49.128798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.129077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.129404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.130171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1f9f3907a6d94b42b40343adca84bba2) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:49.130477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:49.132035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:49.134720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:49.141321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 415/862] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:48 2022] GET /identity/v3/role_assignments?user.id=1f9f3907a6d94b42b40343adca84bba2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:49.147941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800570d2-984a-4df2-91f0-51324af09eff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:49.169758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:49.192695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:49.193741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.194310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.194792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.208189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:49.210386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:49.228366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:49.229417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:49.230396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:49.230396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:49.230396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:49.232391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 448/863] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:49.241307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.307995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.372363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdff86-25bd-453a-aefa-739f78c1ff6e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:49.430406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b4d18-aee5-41ba-99fb-102f10e91ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:49.431036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b4d18-aee5-41ba-99fb-102f10e91ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.431315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b4d18-aee5-41ba-99fb-102f10e91ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.431589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b4d18-aee5-41ba-99fb-102f10e91ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.432247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b4d18-aee5-41ba-99fb-102f10e91ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:49.432551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b4d18-aee5-41ba-99fb-102f10e91ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:49.433978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b4d18-aee5-41ba-99fb-102f10e91ac6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:49.445385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b83e-0f28-4451-85d4-1607d8221ee5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:49.452965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 416/864] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:49.454373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78fed-7117-4ba9-ae48-a5243e5f15d1 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.454722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78fed-7117-4ba9-ae48-a5243e5f15d1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.455002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78fed-7117-4ba9-ae48-a5243e5f15d1 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.456438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 417/865] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:49.459776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78fed-7117-4ba9-ae48-a5243e5f15d1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.489596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d75fb-fa8e-4005-b351-1855c3dac70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:49.490277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d75fb-fa8e-4005-b351-1855c3dac70c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.490563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d75fb-fa8e-4005-b351-1855c3dac70c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.490834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d75fb-fa8e-4005-b351-1855c3dac70c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.491650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d75fb-fa8e-4005-b351-1855c3dac70c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:49.493083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d75fb-fa8e-4005-b351-1855c3dac70c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1f4d75fb-fa8e-4005-b351-1855c3dac70c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:49.502877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:49.503993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:49.504919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 449/866] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:49.505777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec7bf9-1fee-4bc3-8848-e7e2fde44c9d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.509832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec7bf9-1fee-4bc3-8848-e7e2fde44c9d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.509832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec7bf9-1fee-4bc3-8848-e7e2fde44c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.526433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-20ec7bf9-1fee-4bc3-8848-e7e2fde44c9d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:49.527935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-20ec7bf9-1fee-4bc3-8848-e7e2fde44c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:49.532327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:49.557273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:49.586541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78fed-7117-4ba9-ae48-a5243e5f15d1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:49.608261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 450/867] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:49.609359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec7bf9-1fee-4bc3-8848-e7e2fde44c9d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.639617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:49.640742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.641277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.641958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.654966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:49.657746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:49.671860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:49.673102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:49.674304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:49.680811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 418/868] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:49.682208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.764018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec7bf9-1fee-4bc3-8848-e7e2fde44c9d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:49.807576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f241ff-c64a-464b-a1c3-1bf3f28216ac None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:49.812678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:49.813601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.814135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.814608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.815441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:49.816482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:49.818572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:49.830713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 451/869] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:49.832676 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.836653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:49.852810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:49.853436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:49.853755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:49.854034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:49.854988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:49.855111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:49.856513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:49.869375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:49.873731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 419/870] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:49.878256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:49.976058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bae1c-482b-4334-ae98-56ab90d06b40 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:50.010316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e01417-8bd3-49f8-8be0-7d86ab1512f1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:50.015068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:50.015683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:50.015959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:50.016215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:50.016851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:50.017149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:50.018571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:50.029714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 452/871] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:50.029714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:50.066300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59351b-5e51-43a1-a37a-f4f57698a9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:50.067050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59351b-5e51-43a1-a37a-f4f57698a9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:50.067335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59351b-5e51-43a1-a37a-f4f57698a9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:50.067606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59351b-5e51-43a1-a37a-f4f57698a9f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:50.068415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59351b-5e51-43a1-a37a-f4f57698a9f7 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=30f70cdf08d049668d16007e4ca44654) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:50.068805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59351b-5e51-43a1-a37a-f4f57698a9f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:50.071981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59351b-5e51-43a1-a37a-f4f57698a9f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:50.085017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 420/872] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:49 2022] GET /identity/v3/role_assignments?user.id=30f70cdf08d049668d16007e4ca44654&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:50.154776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:50.167361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c8509-96fd-4789-a8f7-8bd8607a4a22 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:50.236426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32facd44-2d48-448a-8f02-372deafe5d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:50.237642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32facd44-2d48-448a-8f02-372deafe5d8b None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:50.238186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32facd44-2d48-448a-8f02-372deafe5d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:50.238646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32facd44-2d48-448a-8f02-372deafe5d8b None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:50.239890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32facd44-2d48-448a-8f02-372deafe5d8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:50.243681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32facd44-2d48-448a-8f02-372deafe5d8b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:50.277260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 453/873] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:52:50 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:51.631056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21aa0f-f6f9-41fa-9f81-24327e3bce0e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:51.631564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21aa0f-f6f9-41fa-9f81-24327e3bce0e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:51.631564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21aa0f-f6f9-41fa-9f81-24327e3bce0e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:51.633357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 421/874] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:51.639148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c546399-1cc1-4fc2-a525-7d938a46ad87 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:51.639657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c546399-1cc1-4fc2-a525-7d938a46ad87 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:51.639760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c546399-1cc1-4fc2-a525-7d938a46ad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:51.662639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2c546399-1cc1-4fc2-a525-7d938a46ad87 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:51.665040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2c546399-1cc1-4fc2-a525-7d938a46ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:51.760622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 454/875] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:51 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:52.083627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21aa0f-f6f9-41fa-9f81-24327e3bce0e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:52.159829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:52.261009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21aa0f-f6f9-41fa-9f81-24327e3bce0e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:52.303158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dc0ac-425a-4eeb-82c5-25ca605fdba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:52.304285 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc0ac-425a-4eeb-82c5-25ca605fdba8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:52.305093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc0ac-425a-4eeb-82c5-25ca605fdba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:52.305093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc0ac-425a-4eeb-82c5-25ca605fdba8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:52.317331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dc0ac-425a-4eeb-82c5-25ca605fdba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ec8dc0ac-425a-4eeb-82c5-25ca605fdba8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:52:52.323718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:52.329108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:52.331253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 422/876] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 12:52:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:52.341803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c546399-1cc1-4fc2-a525-7d938a46ad87 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:52.455591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:52.489237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c546399-1cc1-4fc2-a525-7d938a46ad87 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:52.537974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:52.538671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:52.538946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:52.539382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:52.540261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:52.540669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:52.542563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:52.554774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 455/877] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 12:52:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:52.560235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:52.561147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f266a-c3d4-41de-ac3c-b080c272f189 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:52.561627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f266a-c3d4-41de-ac3c-b080c272f189 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:52.562096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f266a-c3d4-41de-ac3c-b080c272f189 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:52.571440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 423/878] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:52.574276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd570d-fd2b-44e5-bc81-656184f92586 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:52.574745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd570d-fd2b-44e5-bc81-656184f92586 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:52.575253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd570d-fd2b-44e5-bc81-656184f92586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:52.605147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-13dd570d-fd2b-44e5-bc81-656184f92586 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:52.607648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-13dd570d-fd2b-44e5-bc81-656184f92586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:52.693112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b13b3-0bed-4c79-9e52-e11a67a818f5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:52.705722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 424/879] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:52:52.706723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3c5a-a7e6-4413-85a9-abf016e4c484 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:52.707080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3c5a-a7e6-4413-85a9-abf016e4c484 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:52.707430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3c5a-a7e6-4413-85a9-abf016e4c484 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:52.709040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 425/880] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:52.721697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4bd05-2c3b-4e34-ad15-f5c21983065e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:52.721697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4bd05-2c3b-4e34-ad15-f5c21983065e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:52.721697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4bd05-2c3b-4e34-ad15-f5c21983065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:52.743170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c0c4bd05-2c3b-4e34-ad15-f5c21983065e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:52.746123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c0c4bd05-2c3b-4e34-ad15-f5c21983065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:52.770292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:52.771930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:52.772756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:52.773457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:52.786504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:52.788572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:52.798146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:52.802251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:52.809579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:52.810659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:52.811590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:52.811590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:52.811590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:52.813543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 456/881] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:52.819105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:52.885786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 426/882] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:52:52.991935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71402c-fc48-4422-84c2-203c0a28be38 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:53.018684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4bd05-2c3b-4e34-ad15-f5c21983065e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:53.047206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:53.047830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.048155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.048426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.049066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:53.049390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:53.050842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:53.071170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 457/883] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:53.072190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:53.138114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:53.216528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4bd05-2c3b-4e34-ad15-f5c21983065e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:53.224260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d590e-8043-4f41-91a7-dc975140a455 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:53.263710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f471ab-1b09-4b38-9c8d-5ded3657c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:53.264677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f471ab-1b09-4b38-9c8d-5ded3657c084 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.265137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f471ab-1b09-4b38-9c8d-5ded3657c084 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.265591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f471ab-1b09-4b38-9c8d-5ded3657c084 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.266620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f471ab-1b09-4b38-9c8d-5ded3657c084 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:53.268681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f471ab-1b09-4b38-9c8d-5ded3657c084 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:53.270145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f92a5-0e0e-40d5-8986-1ebd0c993972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:53.270859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f92a5-0e0e-40d5-8986-1ebd0c993972 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.271292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f92a5-0e0e-40d5-8986-1ebd0c993972 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.271637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f92a5-0e0e-40d5-8986-1ebd0c993972 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-45f471ab-1b09-4b38-9c8d-5ded3657c084 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:53.280588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:53.282731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:52:53.284838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:53.287404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f92a5-0e0e-40d5-8986-1ebd0c993972 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:53.287605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 427/884] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:52:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:53.287605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddfaca-0f3a-488e-8abe-adb37ddff5b1 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.288020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f92a5-0e0e-40d5-8986-1ebd0c993972 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:53.288218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddfaca-0f3a-488e-8abe-adb37ddff5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.288218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddfaca-0f3a-488e-8abe-adb37ddff5b1 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.288218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 428/885] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:53.288218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddfaca-0f3a-488e-8abe-adb37ddff5b1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-598f92a5-0e0e-40d5-8986-1ebd0c993972 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:52:53.301730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:53.303731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 458/886] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:52:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:53.305629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179ac8f-7888-4e15-a63f-e0c1cc49d291 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.307719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179ac8f-7888-4e15-a63f-e0c1cc49d291 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.307719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179ac8f-7888-4e15-a63f-e0c1cc49d291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.327451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6179ac8f-7888-4e15-a63f-e0c1cc49d291 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:53.328712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6179ac8f-7888-4e15-a63f-e0c1cc49d291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:53.329724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:53.419807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 459/887] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:53 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:53.420917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179ac8f-7888-4e15-a63f-e0c1cc49d291 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:53.469487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddfaca-0f3a-488e-8abe-adb37ddff5b1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:53.474041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:53.527340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:53.528087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.528239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.528482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.529120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:53.530522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:53.553754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 429/888] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:52:53 2022] POST /identity/v3/services => generated 269 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:53.553754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:53.595128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179ac8f-7888-4e15-a63f-e0c1cc49d291 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:53.687016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:53.688230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.688820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.689399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.690743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:53.691386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:53.694402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:53.705697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 460/889] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:52:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:53.715904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:53.723488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:53.757471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1557d-bf25-47c4-9c95-2e1774e853ad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:53.824290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:53.825356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.825951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.826446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.827843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:53.829100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:53.831318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:53.867105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 430/890] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 12:52:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:53.868088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:53.879985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:53.884827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf50934-93cc-4f2b-acdd-9cb266d385d5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:53.933728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:53.935117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:53.936218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:53.936745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:53.938133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:53.938133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:53.939763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:53.947648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 461/891] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:52:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:53.949184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:54.046709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa2291-c995-4747-b4c6-a0734600fece None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:54.119324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:54.120913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:54.121809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:54.123401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:54.124592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:54.128457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:54.148087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:54.150150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:54.151824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:54.157764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 431/892] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:54.161862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:54.166132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c59fe9-7955-4bad-b8d7-e4c7dc93d217 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:54.180609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:54.224245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:54.230876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:54.231610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:54.232097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:54.287640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:54.346452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279b756-20b2-40d5-96cb-9caad29ad215 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:54.371208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:54.373821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:54.395451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:54.396542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:54.397229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:54.398186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:54.399120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:54.399752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:54.401560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:54.412486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 432/893] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:54.414627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:54.544258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:54.549561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 462/894] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:52:53 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:52:54.559381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:54.609402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5e73-3017-4cba-ba52-7ab27ce75a61 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:54.659150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:54.659793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:54.660127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:54.660491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:54.661308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:54.661752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:54.664271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:54.675167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 433/895] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:54.676852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:54.685965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:54.759739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b40593-4ef6-4b3e-a94b-1a61d443e50f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:54.825818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:54.826917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:54.827491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:54.828050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:54.848704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:54.852787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:54.867938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339451-e676-4e21-88e2-4bbe934d1253 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:52:54.884387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:54.885801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:52:54.886907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:52:54.886907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:52:54.886907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:54.887842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 463/896] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:52:54.898121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:54.931984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:54.933077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:54.933580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:54.934131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:54.935067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] RBAC: Adding query filter params (interface=public, service_id=26aebf1ba18540f39382ccb3da6c277b, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:54.935613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:54.937285 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:54.940384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:54.946344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 434/897] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:52:54 2022] GET /identity/v3/endpoints?service_id=26aebf1ba18540f39382ccb3da6c277b&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:54.953765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:55.130975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8488eb-e9cd-4dfd-9171-b5211e6a07aa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:55.155458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181db2f8-adf5-4870-aae5-7cb6f671f7cf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:55.195803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:55.199154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:55.200086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:55.200564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:55.200846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:55.201481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:55.202935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:55.213795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 435/898] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:52:54 2022] GET /identity/v3/services => generated 1268 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 12:52:55.242580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:55.243789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:55.244312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:55.244789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:55.246045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:55.246559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:55.249246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:55.275487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 464/899] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:55.281610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:55.423722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:55.503287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1d009-a4ca-4a6c-80fe-ccf58120f980 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:55.564562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:55.565968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:55.566669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:55.567418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:55.580441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:55.584420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:52:55.607202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:55.609196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:52:55.611237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:52:55.614819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 436/900] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:55.625736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:55.828549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107121ac-0776-484e-8c52-eb5fc51c47a0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:55.896512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a8606-07e8-47f9-83f1-f2b4f4cd42ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:55.897640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a8606-07e8-47f9-83f1-f2b4f4cd42ed None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:55.898156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a8606-07e8-47f9-83f1-f2b4f4cd42ed None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:55.898643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a8606-07e8-47f9-83f1-f2b4f4cd42ed None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:55.900026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a8606-07e8-47f9-83f1-f2b4f4cd42ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:55.900518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a8606-07e8-47f9-83f1-f2b4f4cd42ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:55.903258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a8606-07e8-47f9-83f1-f2b4f4cd42ed None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:55.927787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 465/901] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:55.937559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:56.137733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774f8e8-e5be-4f9a-b638-eba06ef7875c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:56.188058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:56.208928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1cfa5-3de8-4672-bb07-1e1a2953e12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:56.210031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1cfa5-3de8-4672-bb07-1e1a2953e12b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:56.210542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1cfa5-3de8-4672-bb07-1e1a2953e12b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:56.211000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1cfa5-3de8-4672-bb07-1e1a2953e12b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:56.212354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1cfa5-3de8-4672-bb07-1e1a2953e12b None admin] RBAC: Adding query filter params (role.id=f9dfc647115f47a5b0728fc812c34794, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1f9f3907a6d94b42b40343adca84bba2) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:56.212892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1cfa5-3de8-4672-bb07-1e1a2953e12b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:56.215841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1cfa5-3de8-4672-bb07-1e1a2953e12b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:56.233796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 437/902] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:52:55 2022] GET /identity/v3/role_assignments?user.id=1f9f3907a6d94b42b40343adca84bba2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=f9dfc647115f47a5b0728fc812c34794 => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:56.541831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0e20e-4ec0-45d4-a0e7-3ce0dcb2903c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:56.542215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0e20e-4ec0-45d4-a0e7-3ce0dcb2903c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:56.542443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0e20e-4ec0-45d4-a0e7-3ce0dcb2903c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:56.544480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 466/903] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:56.550640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0da9-60ff-4c60-9f58-cfccfab38de7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:56.551030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0da9-60ff-4c60-9f58-cfccfab38de7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:56.551273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0da9-60ff-4c60-9f58-cfccfab38de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:56.588870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fd0b0da9-60ff-4c60-9f58-cfccfab38de7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:56.591061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fd0b0da9-60ff-4c60-9f58-cfccfab38de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:56.723525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 438/904] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:57.083114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ab288-071c-4cf3-9346-c2c38bccda5c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:57.085600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ab288-071c-4cf3-9346-c2c38bccda5c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:57.086528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ab288-071c-4cf3-9346-c2c38bccda5c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:57.089983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 467/905] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:57.100400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eca3c5-264f-4abf-bf93-1a83bed937ab None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:57.102045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eca3c5-264f-4abf-bf93-1a83bed937ab None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:57.102838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eca3c5-264f-4abf-bf93-1a83bed937ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:57.107250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ab288-071c-4cf3-9346-c2c38bccda5c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:57.144112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c9eca3c5-264f-4abf-bf93-1a83bed937ab None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:57.146092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c9eca3c5-264f-4abf-bf93-1a83bed937ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:57.181121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:57.289709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 439/906] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:52:57.299329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ab288-071c-4cf3-9346-c2c38bccda5c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:57.352382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:57.353071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:57.353498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:57.353803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:57.354480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:57.355935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:57.366626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:57.367867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:57.368811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 468/907] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Sep 7 12:52:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:57.377830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eca3c5-264f-4abf-bf93-1a83bed937ab None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:57.456191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:57.505561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eca3c5-264f-4abf-bf93-1a83bed937ab None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:57.544508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:57.545130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:57.545413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:57.545713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:57.546406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:57.546705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:57.548195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:57.554970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 440/908] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:52:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:52:57.564454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:57.580963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:57.729573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:57.735196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9c618-cb3d-467d-9bfd-2230f17115a2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:57.743357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0ac2b-09e3-4eaa-83f2-11c18cf7869a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:57.769177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:57.785544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:57.787446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:57.788081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:57.788625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:57.789758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:57.790619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:57.792939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:57.803235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 469/909] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:52:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:57.813446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:57.822537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:57.823933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:57.824415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:57.824923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:57.827140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:57.830456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:57.862854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 441/910] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:52:57 2022] POST /identity/v3/services => generated 241 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:57.953313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17362cd0-1827-4074-80a8-58be7e471ae6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:58.002812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef109eb-d213-42d7-bd50-d0bccc8f495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:58.003366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef109eb-d213-42d7-bd50-d0bccc8f495e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:58.003647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef109eb-d213-42d7-bd50-d0bccc8f495e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:58.003941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef109eb-d213-42d7-bd50-d0bccc8f495e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:58.004646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef109eb-d213-42d7-bd50-d0bccc8f495e None admin] RBAC: Adding query filter params (interface=public, service_id=88e66cf23e38464f9b13b1d23a3fb811, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:58.004956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef109eb-d213-42d7-bd50-d0bccc8f495e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:58.006881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef109eb-d213-42d7-bd50-d0bccc8f495e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:58.016221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 470/911] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:52:57 2022] GET /identity/v3/endpoints?service_id=88e66cf23e38464f9b13b1d23a3fb811&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:58.023743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:58.153535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a09c98-3475-4f7d-aeac-d5355f5e6feb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:58.198621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e7260-abbf-44df-a2e1-eae40305effb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:58.199353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e7260-abbf-44df-a2e1-eae40305effb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:58.199652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e7260-abbf-44df-a2e1-eae40305effb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:58.199886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e7260-abbf-44df-a2e1-eae40305effb None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:58.200510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e7260-abbf-44df-a2e1-eae40305effb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:58.201946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e7260-abbf-44df-a2e1-eae40305effb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:58.209656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 442/912] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:52:58 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 12:52:58.282266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d58d-f1a7-4116-a0d5-d0e1553ad65d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:58.283067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d58d-f1a7-4116-a0d5-d0e1553ad65d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:58.283067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d58d-f1a7-4116-a0d5-d0e1553ad65d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:58.284396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 471/913] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:52:58.290970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b1d6-0162-42a9-ae71-491993dc929c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:58.291447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b1d6-0162-42a9-ae71-491993dc929c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:58.291716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b1d6-0162-42a9-ae71-491993dc929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:58.315242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a7c2b1d6-0162-42a9-ae71-491993dc929c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:58.317356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a7c2b1d6-0162-42a9-ae71-491993dc929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:58.359021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:58.437785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 443/914] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:52:58.740461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7d58d-f1a7-4116-a0d5-d0e1553ad65d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:58.770932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:58.893338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7d58d-f1a7-4116-a0d5-d0e1553ad65d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:58.964012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:58.965594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:58.966126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:58.966608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:58.967968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:58.971242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:52:58.992916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:52:58.995272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:52:58.997033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 472/915] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:52:59.005543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b1d6-0162-42a9-ae71-491993dc929c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:59.208290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b1d6-0162-42a9-ae71-491993dc929c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:59.258182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e1011-ff1c-4532-981d-03fcdba634c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:59.258799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e1011-ff1c-4532-981d-03fcdba634c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:59.259135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e1011-ff1c-4532-981d-03fcdba634c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:59.259469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e1011-ff1c-4532-981d-03fcdba634c5 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:59.260124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e1011-ff1c-4532-981d-03fcdba634c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:59.260447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e1011-ff1c-4532-981d-03fcdba634c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:59.262125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e1011-ff1c-4532-981d-03fcdba634c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:59.269366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 444/916] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:59.276784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:59.432657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:59.449806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e630a27-7484-4f3e-80b8-c136f26a9042 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:59.450159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e630a27-7484-4f3e-80b8-c136f26a9042 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:59.450325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc1103-eb26-4ddf-996c-38878a065c44 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:59.450486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e630a27-7484-4f3e-80b8-c136f26a9042 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:59.452121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 445/917] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:52:59.458573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed703eb8-6ae9-44b2-9aac-ff3e6cdfaa14 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:59.458880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed703eb8-6ae9-44b2-9aac-ff3e6cdfaa14 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:59.459126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed703eb8-6ae9-44b2-9aac-ff3e6cdfaa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:59.480642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ed703eb8-6ae9-44b2-9aac-ff3e6cdfaa14 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:52:59.482011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed703eb8-6ae9-44b2-9aac-ff3e6cdfaa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:52:59.502806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:59.503426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:59.503811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:59.504102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:59.504946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:52:59.505247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:59.507034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:59.518612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 473/918] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 12:52:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:52:59.519352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:52:59.520323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:52:59.595035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 446/919] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:52:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:52:59.671620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9169-11c5-4931-8f90-5e7372ff05fa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:52:59.714468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:52:59.715096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:52:59.715352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:52:59.715657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:52:59.716432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:52:59.717967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:52:59.744817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 474/920] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:52:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:52:59.981817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed703eb8-6ae9-44b2-9aac-ff3e6cdfaa14 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:00.149162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed703eb8-6ae9-44b2-9aac-ff3e6cdfaa14 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:00.230484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:00.231510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:00.231650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:00.232060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:00.251823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:00.259201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:00.260295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:00.261179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:00.261953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 447/921] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 12:52:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:00.269525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:00.284598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:00.443189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61110e5d-2ff2-4868-91f4-b1972a55f2d2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:00.497630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f2336-d2d1-40e0-b363-f8c2a288a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:00.498647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f2336-d2d1-40e0-b363-f8c2a288a32c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:00.499139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f2336-d2d1-40e0-b363-f8c2a288a32c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:00.499621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f2336-d2d1-40e0-b363-f8c2a288a32c None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:00.501176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f2336-d2d1-40e0-b363-f8c2a288a32c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:00.501761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f2336-d2d1-40e0-b363-f8c2a288a32c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:00.504799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f2336-d2d1-40e0-b363-f8c2a288a32c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:00.522506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 475/922] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 12:53:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:00.528723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:00.550748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:00.669989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab0ad-e68f-4d22-8316-7c3bde58dc8c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:00.722725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:00.723839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:00.724266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:00.724716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:00.748300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:00.751529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:00.790623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:00.792496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:00.795186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:00.795186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:00.795186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:00.795186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 448/923] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:00.804478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2d3da-bb37-455e-bba2-f68d8087c2b1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:00.804478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2d3da-bb37-455e-bba2-f68d8087c2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:00.804478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2d3da-bb37-455e-bba2-f68d8087c2b1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:00.807681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 476/924] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:00.808069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:00.819238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834e431-b469-43d9-9b39-3c51076b586a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:00.820240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834e431-b469-43d9-9b39-3c51076b586a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:00.821085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834e431-b469-43d9-9b39-3c51076b586a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:00.861339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4834e431-b469-43d9-9b39-3c51076b586a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:00.865347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4834e431-b469-43d9-9b39-3c51076b586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:00.958818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:00.990899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:00.997701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 477/925] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:01.044022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd14d53-0a82-46d6-82a7-f5f9f5cabdd1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:01.109047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:01.110104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.110592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.111150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3496bb-2d4c-4ce4-b7a2-b8d8fdf65ef6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.111334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.111953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3496bb-2d4c-4ce4-b7a2-b8d8fdf65ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.112318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:01.112750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3496bb-2d4c-4ce4-b7a2-b8d8fdf65ef6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.112906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:01.120235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 478/926] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:01.120482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:01.122191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7c8b0-c886-4e16-aa4a-76c104ed0fdb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.122706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7c8b0-c886-4e16-aa4a-76c104ed0fdb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.123172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7c8b0-c886-4e16-aa4a-76c104ed0fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.139960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 449/927] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:01.147997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c0e7c8b0-c886-4e16-aa4a-76c104ed0fdb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:01.149429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:01.150166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c0e7c8b0-c886-4e16-aa4a-76c104ed0fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:01.257828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 479/928] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:01.352466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925f20f-2388-4b81-9786-ba6fb9568667 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:01.392085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7c8b0-c886-4e16-aa4a-76c104ed0fdb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:01.405535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:01.420906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:01.422122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.422602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.423054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.437180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:01.439901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:53:01.463659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:01.465502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:01.467394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 450/929] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:53:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:01.477700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:01.525757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7c8b0-c886-4e16-aa4a-76c104ed0fdb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:01.562084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:01.571332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:01.572275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.572713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.573127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.573966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:01.576026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:01.587763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:01.589083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:01.590379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:01.593199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 480/930] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:53:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:01.602855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:01.651036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1b835-949e-4943-82ae-4eda99a2b8fd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:01.707099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:01.708310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.708791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.709401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.710533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:01.710898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:01.712370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:01.743675 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 451/931] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 12:53:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:01.743675 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:01.778859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb66dc-98fe-4e02-b89a-2df9ad6b6cad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:01.821822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:01.834286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:01.835366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.835829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.836281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.837387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:01.837943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:01.840481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:01.852309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 481/932] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:01.854051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:01.924016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aed210-a19e-4bd2-bb55-00d7c9d895a6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:01.966628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:01.991012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:01.992406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:01.993116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:01.993831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:01.995092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:01.998044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:02.015914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:02.018111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 12:53:02.020122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:02.024829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 452/933] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Sep 7 12:53:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:02.026592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:02.058139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92314ad-46dd-496c-b9b2-fd936003c49e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:02.124730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:02.126028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:02.126513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:02.126979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:02.148094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:02.150729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:02.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:02.179620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:02.182491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:02.182491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:02.182491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:02.182491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 482/934] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:02.182491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:02.227570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c794c7-5e14-46ac-b837-23c30d02db75 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:02.248589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:02.294994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:02.296251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:02.296544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:02.297019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:02.298430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:02.298975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:02.301613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:02.317414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 453/935] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:02.317414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:02.361628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904f5b1-9fde-417e-931a-e433369bcb58 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:02.388090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:02.429114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:02.430251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:02.430741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:02.431220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:02.432350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:02.432873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:02.435803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:02.447006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 483/936] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:53:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:02.448518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:02.505659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6db24-da21-4bd0-9035-f502af135ae1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:02.557730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:02.558367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:02.558741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:02.559013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:02.559614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:02.560063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:02.561695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:02.585005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 454/937] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:02.585005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:02.637608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:02.650560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1d3a5-8edc-4551-a8a4-3865cec8362a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:02.713602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:02.714648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:02.715125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:02.715590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:02.716875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] RBAC: Adding query filter params (interface=public, service_id=3e9ad26cb56147c4abc838eae23de5e5, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:02.717404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:02.720029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:02.736953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 484/938] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:53:02 2022] GET /identity/v3/endpoints?service_id=3e9ad26cb56147c4abc838eae23de5e5&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:02.736953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:02.754362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1b69-8264-46fd-83af-81ecf7bee3f7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:02.804792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:02.811542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:02.812595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:02.813038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:02.813486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:02.814572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:02.815030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:02.817374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:02.824748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 455/939] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:53:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:02.832431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:02.924887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189925b5-1020-43ff-9164-55cd174757cb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:02.995168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:02.996230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:02.996598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:02.997042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:03.006261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:03.009045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:03.011011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:03.016374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996e0c6-1e93-4465-a0ec-5120779ac16d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:53:03.030560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:03.032956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 485/940] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:03.035122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:03.092759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:03.094148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:03.094316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:03.094745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:03.095818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:03.098423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:03.117714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 456/941] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:53:02 2022] GET /identity/v3/services => generated 1497 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 12:53:03.117714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:03.202943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:03.209720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6529a2d-22f6-491c-8d57-ef5b331667e9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:03.234141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa011db-4890-459c-85d4-06fdb62d925c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:03.264067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2dd09-b511-4b37-9c44-480713d290c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:03.264885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2dd09-b511-4b37-9c44-480713d290c8 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:03.265282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2dd09-b511-4b37-9c44-480713d290c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:03.265854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2dd09-b511-4b37-9c44-480713d290c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:03.266773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2dd09-b511-4b37-9c44-480713d290c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:03.268703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2dd09-b511-4b37-9c44-480713d290c8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:03.301729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 486/942] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:53:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:03.301729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b3bb1-d698-4c76-92b6-c575aa47e280 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:03.301729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b3bb1-d698-4c76-92b6-c575aa47e280 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:03.301729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b3bb1-d698-4c76-92b6-c575aa47e280 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:03.309722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 487/943] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:03.313612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a5c2-8228-489e-b433-7d23a013755a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:03.314194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a5c2-8228-489e-b433-7d23a013755a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:03.314699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a5c2-8228-489e-b433-7d23a013755a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:03.316038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:03.316965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:03.317414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:03.317882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:03.319114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:03.319610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:03.322126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:03.352704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-61e9a5c2-8228-489e-b433-7d23a013755a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:03.352979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 457/944] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:53:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:03.355133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-61e9a5c2-8228-489e-b433-7d23a013755a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:03.367427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:03.423880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:03.485932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 488/945] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:03.573817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7525e7-e786-456d-b668-09dbaeb7a9d9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:03.619703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:03.620822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:03.621392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:03.621897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1f9f3907a6d94b42b40343adca84bba2/roles/f9dfc647115f47a5b0728fc812c34794` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:03.625971 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=1f9f3907a6d94b42b40343adca84bba2, role_id=f9dfc647115f47a5b0728fc812c34794)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:03.627843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:03.649381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 458/946] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:53:03 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1f9f3907a6d94b42b40343adca84bba2/roles/f9dfc647115f47a5b0728fc812c34794 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:53:03.941963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9a5c2-8228-489e-b433-7d23a013755a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:04.078383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:04.117474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9a5c2-8228-489e-b433-7d23a013755a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:04.174177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:04.174829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:04.175111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:04.175382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:04.176046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:04.177977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:04.182353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING keystone.server.flask.common [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 12:53:04.215360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 12:53:04.215360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 12:53:04.215360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [parameters: {'id': '7993b8e20cc5403a9b9d96581678faad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 12:53:04.215360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7993b8e20cc5403a9b9d96581678faad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7993b8e20cc5403a9b9d96581678faad. Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 12:53:04.251104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 12:53:04.252625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Sep 07 12:53:04.254474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 12:53:04.256196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 12:53:04.262296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 12:53:04.264528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'id': '7993b8e20cc5403a9b9d96581678faad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:04.266641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:04.268367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7993b8e20cc5403a9b9d96581678faad. Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:04.274631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 489/947] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Sep 7 12:53:03 2022] POST /identity/v3/projects => generated 262 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:04.275571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:04.394133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:04.424045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d48fc-e9b7-4d2d-bef0-dc8a8d17bd06 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:04.468265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38859d86-89ad-49e9-9e42-9f228ef34dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:04.468815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38859d86-89ad-49e9-9e42-9f228ef34dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:04.469157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38859d86-89ad-49e9-9e42-9f228ef34dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:04.469428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38859d86-89ad-49e9-9e42-9f228ef34dd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:04.482094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38859d86-89ad-49e9-9e42-9f228ef34dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:04.483762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38859d86-89ad-49e9-9e42-9f228ef34dd5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-38859d86-89ad-49e9-9e42-9f228ef34dd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:53:04.504021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:04.507650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:04.514286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 459/948] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:04.519226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:04.703448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d06fdc-84ba-4dd6-9a2e-0316ba79a710 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:04.779001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167ff3b-8f84-4067-a94b-951e483347d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:04.779871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167ff3b-8f84-4067-a94b-951e483347d8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:04.780417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167ff3b-8f84-4067-a94b-951e483347d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:04.780703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167ff3b-8f84-4067-a94b-951e483347d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:04.783207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167ff3b-8f84-4067-a94b-951e483347d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:04.783768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167ff3b-8f84-4067-a94b-951e483347d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:04.788110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167ff3b-8f84-4067-a94b-951e483347d8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:04.812476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:04.821444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 490/949] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:53:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:05.929692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1d43-1ab7-41c0-8bc4-8ae28a38dcef None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:05.930457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1d43-1ab7-41c0-8bc4-8ae28a38dcef None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:05.930457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1d43-1ab7-41c0-8bc4-8ae28a38dcef None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:05.931766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 460/950] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:05.937984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2e725-f057-4c90-94fa-308a35d6a015 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:05.938969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2e725-f057-4c90-94fa-308a35d6a015 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:05.939447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2e725-f057-4c90-94fa-308a35d6a015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:05.964526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2ea2e725-f057-4c90-94fa-308a35d6a015 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:05.966974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2ea2e725-f057-4c90-94fa-308a35d6a015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:06.088708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 491/951] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:06.455532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f1d43-1ab7-41c0-8bc4-8ae28a38dcef None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:06.606521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f1d43-1ab7-41c0-8bc4-8ae28a38dcef None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:06.675600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b1f3-bb31-4c7e-afd8-b0b999e4b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:06.676718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b1f3-bb31-4c7e-afd8-b0b999e4b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:06.677200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b1f3-bb31-4c7e-afd8-b0b999e4b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:06.677708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b1f3-bb31-4c7e-afd8-b0b999e4b0a9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:06.679631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902b1f3-bb31-4c7e-afd8-b0b999e4b0a9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:06.682507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902b1f3-bb31-4c7e-afd8-b0b999e4b0a9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c902b1f3-bb31-4c7e-afd8-b0b999e4b0a9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:06.702324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:06.704206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:06.706874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 461/952] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:53:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:06.719922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2e725-f057-4c90-94fa-308a35d6a015 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:06.851604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85480938-a4ae-4a17-84f4-7db5cb69ffc2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:06.852044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85480938-a4ae-4a17-84f4-7db5cb69ffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:06.852417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85480938-a4ae-4a17-84f4-7db5cb69ffc2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:06.861115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 462/953] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:06.863549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d686ea5-0bd4-4056-8686-7b58d1a7d0ca None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:06.864050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d686ea5-0bd4-4056-8686-7b58d1a7d0ca None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:06.864451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d686ea5-0bd4-4056-8686-7b58d1a7d0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:06.873724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:06.889098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6d686ea5-0bd4-4056-8686-7b58d1a7d0ca None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:06.890531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6d686ea5-0bd4-4056-8686-7b58d1a7d0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:06.918953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2e725-f057-4c90-94fa-308a35d6a015 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:06.965553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:06.973988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4091ace-bdad-4ec0-87d3-8e24c40ff1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:06.974618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4091ace-bdad-4ec0-87d3-8e24c40ff1af None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:06.974933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4091ace-bdad-4ec0-87d3-8e24c40ff1af None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:06.975259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4091ace-bdad-4ec0-87d3-8e24c40ff1af None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:06.976757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4091ace-bdad-4ec0-87d3-8e24c40ff1af None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:06.977040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4091ace-bdad-4ec0-87d3-8e24c40ff1af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:06.978504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4091ace-bdad-4ec0-87d3-8e24c40ff1af None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:06.989230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 492/954] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:06.989230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9306f1-84a4-491f-b53d-846b4e53ce3a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:06.989230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9306f1-84a4-491f-b53d-846b4e53ce3a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:06.989230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9306f1-84a4-491f-b53d-846b4e53ce3a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:06.989230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 493/955] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:06.990428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9306f1-84a4-491f-b53d-846b4e53ce3a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:07.008705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 463/956] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:07.010781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a84ee-5357-4ce3-8c05-f729ef56526f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:07.011320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a84ee-5357-4ce3-8c05-f729ef56526f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:07.011799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a84ee-5357-4ce3-8c05-f729ef56526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:07.047674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c48a84ee-5357-4ce3-8c05-f729ef56526f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:07.050188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c48a84ee-5357-4ce3-8c05-f729ef56526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:07.162037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9306f1-84a4-491f-b53d-846b4e53ce3a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:07.182454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 464/957] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:07.220797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:07.221629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:07.222112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:07.222564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:07.223618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:07.224087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:07.226499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:07.235467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 494/958] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:07.245289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a84ee-5357-4ce3-8c05-f729ef56526f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:07.390273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:07.418900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:07.460706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a84ee-5357-4ce3-8c05-f729ef56526f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:07.526839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2118228-b6d5-4c03-ba1b-dab18f234a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:07.527698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2118228-b6d5-4c03-ba1b-dab18f234a58 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:07.528161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2118228-b6d5-4c03-ba1b-dab18f234a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:07.528468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2118228-b6d5-4c03-ba1b-dab18f234a58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:07.528975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2118228-b6d5-4c03-ba1b-dab18f234a58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:07.530410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2118228-b6d5-4c03-ba1b-dab18f234a58 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:07.530803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:07.573939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 465/959] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:53:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:07.576220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865ea94-6a88-43fb-a255-12128e5b6f1f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:07.576666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865ea94-6a88-43fb-a255-12128e5b6f1f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:07.577109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865ea94-6a88-43fb-a255-12128e5b6f1f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:07.585700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 466/960] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:07.588408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd890c5-b8e4-4a34-9a68-b8d5cd0277be None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:07.588905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd890c5-b8e4-4a34-9a68-b8d5cd0277be None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:07.589272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd890c5-b8e4-4a34-9a68-b8d5cd0277be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:07.617492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2cd890c5-b8e4-4a34-9a68-b8d5cd0277be None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:07.618134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19dab1-9c8a-4306-ba33-170f28e69e44 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:07.619163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2cd890c5-b8e4-4a34-9a68-b8d5cd0277be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:07.688409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:07.689470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:07.689991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:07.690465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:07.691589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:07.694472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:07.711326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:07.712936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:07.714207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:07.719974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 495/961] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:07.721118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:07.738595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 467/962] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:07.738595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd890c5-b8e4-4a34-9a68-b8d5cd0277be None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:07.856700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:07.899169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1013f-0847-4d41-a0ca-a1ead4b46383 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:07.920171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:07.939889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:07.940701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:07.941072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:07.941431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:07.953799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:07.960673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd890c5-b8e4-4a34-9a68-b8d5cd0277be None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:07.960886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:07.962929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:07.965259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 496/963] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 12:53:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:07.973150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.011067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:08.012493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:08.012493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:08.012953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:08.013189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:08.013471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:08.016427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:08.038009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 468/964] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:53:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:08.041061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.138892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038413bb-9866-4ffc-bbc8-86bf0d209f96 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:08.184829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:08.185454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:08.185791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:08.186026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:08.186899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:08.187059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:08.188688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:08.191106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234086e-94ec-4622-8d68-3f0f20e5f39b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:08.198894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 497/965] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 12:53:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:08.199509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.215567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:08.245193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:08.246014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:08.246304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:08.246609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:08.247463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:08.247941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:08.250676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:08.256500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:08.270051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 469/966] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:53:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:08.270051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.359250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e2a50-50c3-4e0f-b268-49448c4e7e90 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:08.413143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:08.414136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:08.414424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:08.414691 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:08.415446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:08.417293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:08.425332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf021e4-2af1-4d94-bd56-1c8910ec157d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:08.438923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 12:53:08.438923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 12:53:08.438923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [parameters: {'user_id': '4452b4a10e484acd87837b1fc53058d5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 12:53:08.438923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Sep 07 12:53:08.449253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 12:53:08.450408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 12:53:08.452453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 12:53:08.454371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 12:53:08.462536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 12:53:08.464390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'user_id': '4452b4a10e484acd87837b1fc53058d5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:08.466079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 12:53:08.467742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 498/967] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 12:53:08 2022] POST /identity/v3/users => generated 160 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:08.478579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.490618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:08.492100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:08.492639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:08.493147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:08.511764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:08.514617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:08.540561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:08.543381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:08.546197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:08.546197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:08.546197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:08.546197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 470/968] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:08.555397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.605043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:08.629046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09e8c-b485-4cfb-9d58-41efd25b5455 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:08.632368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:08.702825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:08.703996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:08.704482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:08.704947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:08.721186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:08.724542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:53:08.749238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:08.751511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 12:53:08.753429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:08.758276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 499/969] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:53:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:08.762412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.795649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb408543-6084-4d67-b671-281b28f53011 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:08.884840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:08.885555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:08.886070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:08.886541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:08.887977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:08.888703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:08.891216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:08.921703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 471/970] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:08.927560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:08.983470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537a192-74bb-4446-ba28-40d5ac81ab29 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:09.027110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:09.027755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:09.028033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:09.028302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:09.029032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:09.029337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:09.030898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:09.074717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 500/971] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 12:53:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:09.118128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:09.145101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736aaf2-8fb1-4229-be71-034d4a759a92 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:09.219402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:09.220520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:09.220998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:09.221474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:09.236758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:09.239707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:53:09.266738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:09.267801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 12:53:09.268805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:09.269997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 472/972] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:53:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:09.279431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:09.327239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:09.498541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0accb94-06df-430a-85ef-e5c22a0869c2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:09.576591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:09.577420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:09.577919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:09.578325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:09.579264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:09.579714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:09.581586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:09.621388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 501/973] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 12:53:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:09.631225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:09.817980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e901f9f-c32c-469b-9a39-d967d2dedd9d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:09.871517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7a81a-bf99-4c80-9117-186f98558ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:09.872570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7a81a-bf99-4c80-9117-186f98558ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:09.873087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7a81a-bf99-4c80-9117-186f98558ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:09.873559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7a81a-bf99-4c80-9117-186f98558ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:09.907884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:10.002156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7a81a-bf99-4c80-9117-186f98558ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:10.003858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7a81a-bf99-4c80-9117-186f98558ce7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:10.171155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 473/974] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:09 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:53:10.180073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:10.311025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:10.340351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb468a0-4805-4823-89b4-092cd66ad607 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:10.373428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205d02-1d0b-4ee9-a356-f4e680b3047c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:10.374144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205d02-1d0b-4ee9-a356-f4e680b3047c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:10.374593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205d02-1d0b-4ee9-a356-f4e680b3047c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:10.376968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 474/975] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:10.386704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30a0c-2fbc-4cd0-986f-a8f7e312469a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:10.387355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30a0c-2fbc-4cd0-986f-a8f7e312469a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:10.387810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30a0c-2fbc-4cd0-986f-a8f7e312469a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:10.387993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:10.388688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:10.389002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:10.389306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:10.404943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:10.407371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:10.418830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-34e30a0c-2fbc-4cd0-986f-a8f7e312469a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:10.421404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-34e30a0c-2fbc-4cd0-986f-a8f7e312469a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:10.428093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:10.430042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:10.437419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:10.437419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:10.437419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:10.437419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 502/976] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:10.437419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:10.509379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:10.555523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 475/977] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:10.624941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ff08e-02a7-4b69-8ca6-c81e154c5baa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:10.662501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:10.677562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:10.678553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:10.680541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:10.680541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:10.681917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:10.681917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:10.683404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:10.706056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 503/978] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:10.709832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e30a0c-2fbc-4cd0-986f-a8f7e312469a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:10.914432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e30a0c-2fbc-4cd0-986f-a8f7e312469a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:10.951816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:10.984558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:10.985595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:10.986036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:10.986449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.001425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:11.003923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:53:11.024713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:11.027693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:11.029197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 476/979] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:11.036516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:11.110054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:11.157987 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1df12-7c03-4822-a23d-37ee12e1d71f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:11.209307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f992b-7d26-41b4-81b4-786707d50ee5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:11.227886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:11.229333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:11.229510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:11.229915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.231021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:11.233611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:11.252305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:11.253966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:11.254853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 504/980] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:53:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:11.265173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:11.281598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02d15d-2593-4866-82bf-4db0aae80e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:11.282702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d15d-2593-4866-82bf-4db0aae80e4c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:11.283163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d15d-2593-4866-82bf-4db0aae80e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:11.283602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d15d-2593-4866-82bf-4db0aae80e4c None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.284888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d15d-2593-4866-82bf-4db0aae80e4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:11.285430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d15d-2593-4866-82bf-4db0aae80e4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:11.288402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d15d-2593-4866-82bf-4db0aae80e4c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:11.311865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 477/981] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:53:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:11.314138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c02a-f50a-42e5-a749-00887216ae15 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:11.314669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c02a-f50a-42e5-a749-00887216ae15 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:11.315127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c02a-f50a-42e5-a749-00887216ae15 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.317980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 478/982] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:11.319330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c02a-f50a-42e5-a749-00887216ae15 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:11.372050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:11.449791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e1a63-e2ba-42aa-add8-e0551907768a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:11.485537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:11.493581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c02a-f50a-42e5-a749-00887216ae15 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:11.513278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242b965-18d5-479b-9064-4fd4de02e5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:11.514333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242b965-18d5-479b-9064-4fd4de02e5bd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:11.514814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242b965-18d5-479b-9064-4fd4de02e5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:11.515306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242b965-18d5-479b-9064-4fd4de02e5bd None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.516458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242b965-18d5-479b-9064-4fd4de02e5bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:11.516993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242b965-18d5-479b-9064-4fd4de02e5bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:11.519662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242b965-18d5-479b-9064-4fd4de02e5bd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:11.529472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 505/983] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:11.531893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c40ff-e676-439c-8f11-41ee82e18b15 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:11.532393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c40ff-e676-439c-8f11-41ee82e18b15 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:11.532868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c40ff-e676-439c-8f11-41ee82e18b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.550046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:11.550644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:11.550925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:11.551195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.551935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] RBAC: Adding query filter params (role.id=f9dfc647115f47a5b0728fc812c34794, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1f9f3907a6d94b42b40343adca84bba2) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:11.552245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:11.555350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:11.562121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 479/984] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:53:11 2022] GET /identity/v3/role_assignments?user.id=1f9f3907a6d94b42b40343adca84bba2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=f9dfc647115f47a5b0728fc812c34794 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:11.562969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:11.564219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fe9c40ff-e676-439c-8f11-41ee82e18b15 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:11.567628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fe9c40ff-e676-439c-8f11-41ee82e18b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:11.709821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 506/985] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:11.747164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad6cd-e26c-4560-8251-017c9a6da816 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:11.815724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32059286-8880-43fb-9725-14126e6901ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:11.816758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32059286-8880-43fb-9725-14126e6901ac None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:11.817291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32059286-8880-43fb-9725-14126e6901ac None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:11.817828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32059286-8880-43fb-9725-14126e6901ac None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:11.819189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32059286-8880-43fb-9725-14126e6901ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:11.819918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32059286-8880-43fb-9725-14126e6901ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:11.822481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32059286-8880-43fb-9725-14126e6901ac None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:11.832202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 480/986] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:11.842362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c40ff-e676-439c-8f11-41ee82e18b15 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:11.926884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:11.998646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c40ff-e676-439c-8f11-41ee82e18b15 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:12.047427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:12.048874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:12.049596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:12.050382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:12.051856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] RBAC: Adding query filter params (interface=admin, service_id=3e9ad26cb56147c4abc838eae23de5e5, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:12.052686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:12.056334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:12.065648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 507/987] 10.222.0.17 () {56 vars in 1249 bytes} [Wed Sep 7 12:53:11 2022] GET /identity/v3/endpoints?service_id=3e9ad26cb56147c4abc838eae23de5e5&interface=admin®ion_id=RegionOne => generated 193 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:12.071990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32059286-8880-43fb-9725-14126e6901ac None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:12.113973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:12.204478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:12.235693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32059286-8880-43fb-9725-14126e6901ac None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:12.291901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6861c4-0331-40d0-8807-d5c6b7033f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:12.292911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6861c4-0331-40d0-8807-d5c6b7033f3c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:12.293385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6861c4-0331-40d0-8807-d5c6b7033f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:12.294010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6861c4-0331-40d0-8807-d5c6b7033f3c None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:12.294980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6861c4-0331-40d0-8807-d5c6b7033f3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:12.297803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6861c4-0331-40d0-8807-d5c6b7033f3c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:12.317793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 481/988] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:53:12 2022] GET /identity/v3/services => generated 1497 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 12:53:12.377569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2fcc-81c4-4560-b8f0-ff6a268d78c8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:12.403771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:12.440942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603edd4d-0233-44eb-9848-fe47a95fe3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:12.441940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd4d-0233-44eb-9848-fe47a95fe3c3 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:12.442369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd4d-0233-44eb-9848-fe47a95fe3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:12.442745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd4d-0233-44eb-9848-fe47a95fe3c3 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:12.444023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603edd4d-0233-44eb-9848-fe47a95fe3c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:12.446287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603edd4d-0233-44eb-9848-fe47a95fe3c3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:12.468523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 508/989] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:53:12 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:14.466974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939adac-8a6d-4f4f-ad8e-bd9f5639d214 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:14.467777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939adac-8a6d-4f4f-ad8e-bd9f5639d214 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:14.467777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939adac-8a6d-4f4f-ad8e-bd9f5639d214 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:14.468954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 482/990] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:14.478218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3ce4-9b1e-4ec8-9f6f-f7999bc834da None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:14.478920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3ce4-9b1e-4ec8-9f6f-f7999bc834da None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:14.479218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3ce4-9b1e-4ec8-9f6f-f7999bc834da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:14.504298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a51d3ce4-9b1e-4ec8-9f6f-f7999bc834da None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:14.505782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a51d3ce4-9b1e-4ec8-9f6f-f7999bc834da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:14.650525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 509/991] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:14.976099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939adac-8a6d-4f4f-ad8e-bd9f5639d214 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:15.018130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc4ef2-a09d-422d-83ed-3665bf986d17 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.018386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc4ef2-a09d-422d-83ed-3665bf986d17 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.018660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc4ef2-a09d-422d-83ed-3665bf986d17 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.020234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 510/992] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:15.027328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273012d1-541d-4ac1-81ea-15cf554c6bf3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.027596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273012d1-541d-4ac1-81ea-15cf554c6bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.027877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273012d1-541d-4ac1-81ea-15cf554c6bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.066476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-273012d1-541d-4ac1-81ea-15cf554c6bf3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:15.069146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-273012d1-541d-4ac1-81ea-15cf554c6bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:15.143235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:15.162538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939adac-8a6d-4f4f-ad8e-bd9f5639d214 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:15.212968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 511/993] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:15.239235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85569b1d-0ea0-4359-9467-dc9269c240b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:15.240239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85569b1d-0ea0-4359-9467-dc9269c240b4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.240746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85569b1d-0ea0-4359-9467-dc9269c240b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.241294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85569b1d-0ea0-4359-9467-dc9269c240b4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.242403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85569b1d-0ea0-4359-9467-dc9269c240b4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:15.245331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85569b1d-0ea0-4359-9467-dc9269c240b4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-85569b1d-0ea0-4359-9467-dc9269c240b4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:15.264830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:15.266640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:15.267847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:15.274191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 483/994] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Sep 7 12:53:14 2022] GET /identity/v3/services/image => generated 86 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:15.278338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273012d1-541d-4ac1-81ea-15cf554c6bf3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:15.323365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:15.442627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0e8f-3588-4f37-a9c1-7477cdb7d1d8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.442967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0e8f-3588-4f37-a9c1-7477cdb7d1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.443183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0e8f-3588-4f37-a9c1-7477cdb7d1d8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.444996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 484/995] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:15.451448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3729f4-2027-4d89-a614-17e1a199a074 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.451746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3729f4-2027-4d89-a614-17e1a199a074 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.451997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3729f4-2027-4d89-a614-17e1a199a074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.476925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7d3729f4-2027-4d89-a614-17e1a199a074 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:15.478645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7d3729f4-2027-4d89-a614-17e1a199a074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:15.499618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273012d1-541d-4ac1-81ea-15cf554c6bf3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:15.557261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:15.557933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.558281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.558531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.559117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] RBAC: Adding query filter params (name=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:15.559426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:15.560862 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:15.567811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 512/996] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:15.568770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:15.607883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 485/997] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:15.609529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3729f4-2027-4d89-a614-17e1a199a074 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:15.701633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:15.715523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dc286-8750-4677-b35b-8b095c193011 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:15.765964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:15.766806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.767265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.767555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.768167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:15.769593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:15.781999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:15.783956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:15.785861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:15.788158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 513/998] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 12:53:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:15.795398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:15.815361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3729f4-2027-4d89-a614-17e1a199a074 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:15.879712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:15.880931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:15.881456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:15.881976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:15.883098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] RBAC: Adding query filter params (type=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:15.883705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:15.886257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:15.897230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 486/999] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:15.909558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:16.015818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea550ad3-1688-475d-8993-bd18af8eb824 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:16.095222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:16.096045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:16.097244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:16.097803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:16.099351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:16.100325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:16.103250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:16.114534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 514/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:16.122818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d7b3c-b975-4a8c-91fb-437cf85a272e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:16.124365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:16.166189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:16.184234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:16.185326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:16.185857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:16.186315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:16.187348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:16.190006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:16.207562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:16.209553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:16.211326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:16.214002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 487/1001] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:16.218731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:16.249976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:16.292943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0554-415c-4bda-8218-af0a996e9718 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:16.351928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:16.352923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:16.353461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:16.354053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:16.355272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:16.355866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:16.358697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:16.369714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 515/1002] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 12:53:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:16.376079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:16.442714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9228ac5-4281-435c-9e02-e628cd42e861 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:16.504992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:16.506430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:16.506791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:16.507260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:16.508426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:16.508948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:16.511698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:16.521764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 488/1003] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:53:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:16.530014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:16.560403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:16.593321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa98d4-d250-471a-9119-06b179cb9a6d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:16.646350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:16.647614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:16.647614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:16.647614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:16.648966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:16.650435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:16.651063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:16.676268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 516/1004] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:53:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:53:16.705564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d212144-bdb2-43e2-8311-c7e8443ea436 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:16.792475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:16.793531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:16.794143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:16.794566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:16.795892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] RBAC: Adding query filter params (type=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:16.796363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:16.798995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:16.809416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 489/1005] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:53:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:16.819600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:17.003331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756c553-f48f-4d34-a9c6-e188677d5432 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:17.068350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9790a-3626-43a4-b046-2bab7cd56245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:17.069485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9790a-3626-43a4-b046-2bab7cd56245 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:17.071126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9790a-3626-43a4-b046-2bab7cd56245 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:17.071126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9790a-3626-43a4-b046-2bab7cd56245 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:17.072401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9790a-3626-43a4-b046-2bab7cd56245 None admin] RBAC: Adding query filter params (interface=public, service_id=b7b5da4ab292493d9af4f70f3cdf58ea, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:17.072401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9790a-3626-43a4-b046-2bab7cd56245 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:17.074531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9790a-3626-43a4-b046-2bab7cd56245 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:17.084484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 517/1006] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:53:16 2022] GET /identity/v3/endpoints?service_id=b7b5da4ab292493d9af4f70f3cdf58ea&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:17.090940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:17.239741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24923ab1-3fba-40e8-b6d2-bc5b586521ec None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:17.300976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6018b6-364c-4f36-a0ca-b28424e935e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:17.301823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6018b6-364c-4f36-a0ca-b28424e935e9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:17.302231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6018b6-364c-4f36-a0ca-b28424e935e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:17.302633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6018b6-364c-4f36-a0ca-b28424e935e9 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:17.303800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6018b6-364c-4f36-a0ca-b28424e935e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:17.306381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6018b6-364c-4f36-a0ca-b28424e935e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:17.309928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:17.319786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 490/1007] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:53:17 2022] GET /identity/v3/services => generated 1727 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 12:53:19.366480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96b205-c6ab-459c-8bcd-25120c3a32cb None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:19.367314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96b205-c6ab-459c-8bcd-25120c3a32cb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:19.367314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96b205-c6ab-459c-8bcd-25120c3a32cb None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:19.369739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 518/1008] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:19.389015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dcb87-3463-4951-9280-c00a96ab34d1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:19.389015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dcb87-3463-4951-9280-c00a96ab34d1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:19.389015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dcb87-3463-4951-9280-c00a96ab34d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:19.420566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ab4dcb87-3463-4951-9280-c00a96ab34d1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:19.422493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ab4dcb87-3463-4951-9280-c00a96ab34d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:19.527622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 491/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:19.550574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759292b-61db-4d6d-bf46-fca32a261188 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:19.550840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759292b-61db-4d6d-bf46-fca32a261188 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:19.551200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759292b-61db-4d6d-bf46-fca32a261188 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:19.553011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 519/1010] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:19.559904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0deb-af2e-4972-97c0-c83ff2cf02dc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:19.560208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0deb-af2e-4972-97c0-c83ff2cf02dc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:19.560530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0deb-af2e-4972-97c0-c83ff2cf02dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:19.582542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-763f0deb-af2e-4972-97c0-c83ff2cf02dc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:19.583885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-763f0deb-af2e-4972-97c0-c83ff2cf02dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:19.705724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 492/1011] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:19.812503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759292b-61db-4d6d-bf46-fca32a261188 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:19.826909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:19.831935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897de0f-e1b0-4ffb-b1b5-09ac7c01b74e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:19.832269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897de0f-e1b0-4ffb-b1b5-09ac7c01b74e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:19.832555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897de0f-e1b0-4ffb-b1b5-09ac7c01b74e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:19.834277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 493/1012] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:19.841152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd55d4-8b78-4d3d-b513-068f721e009f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:19.841600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd55d4-8b78-4d3d-b513-068f721e009f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:19.842066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd55d4-8b78-4d3d-b513-068f721e009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:19.869188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cccd55d4-8b78-4d3d-b513-068f721e009f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:19.870472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cccd55d4-8b78-4d3d-b513-068f721e009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:19.957993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:19.979200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 494/1013] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:20.005505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759292b-61db-4d6d-bf46-fca32a261188 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:20.072592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:20.073420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:20.073876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:20.074275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:20.075091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:20.076678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:20.104167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 520/1014] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:53:19 2022] POST /identity/v3/services => generated 242 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:20.112240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd55d4-8b78-4d3d-b513-068f721e009f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:20.299579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd55d4-8b78-4d3d-b513-068f721e009f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:20.388486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:20.389153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:20.389992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:20.389992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:20.391129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:20.392640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:20.400242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 495/1015] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:20.403782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:20.445167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:20.525443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:20.537542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:20.607710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6aa5c-ac0f-42ef-970c-13154c888bdb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:20.618709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b416e-606c-4e41-8889-c3ee19a36167 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:20.667104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:20.667780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:20.668045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:20.668348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:20.668901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:20.670345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:20.671832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:20.672869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:20.673857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:20.674793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:20.675991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:20.678846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:20.681717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:20.683621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:20.684535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 496/1016] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:20.691469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:20.729737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 521/1017] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:53:20 2022] POST /identity/v3/projects => generated 311 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:20.888794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749691b8-f706-4dcb-b77a-1330c647e2eb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:20.922258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:20.959487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:20.960452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:20.960865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:20.961271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:20.962336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:20.962829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:20.965294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:20.974961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 497/1018] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:53:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:20.983960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:21.184176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40691ebe-94f1-4058-ac6a-7612c4e56eb5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:21.216334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:21.243883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f837f-d202-4540-abfc-747357357750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:21.244868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f837f-d202-4540-abfc-747357357750 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:21.245312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f837f-d202-4540-abfc-747357357750 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:21.245783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f837f-d202-4540-abfc-747357357750 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:21.246856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f837f-d202-4540-abfc-747357357750 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:21.247343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f837f-d202-4540-abfc-747357357750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:21.248850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f837f-d202-4540-abfc-747357357750 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:21.256274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 522/1019] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 12:53:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:21.263835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:21.458219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10c0a7-8d21-421b-ab72-6e1dbeb245b1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:21.528214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf00886-9e5e-4f1b-af2a-b037ea983110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:21.529298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf00886-9e5e-4f1b-af2a-b037ea983110 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:21.529824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf00886-9e5e-4f1b-af2a-b037ea983110 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:21.530292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf00886-9e5e-4f1b-af2a-b037ea983110 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:21.531542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf00886-9e5e-4f1b-af2a-b037ea983110 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:21.534066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf00886-9e5e-4f1b-af2a-b037ea983110 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:21.593711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 498/1020] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:53:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:22.843852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba281d-22eb-40e3-8397-836307b0da62 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:22.844692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba281d-22eb-40e3-8397-836307b0da62 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:22.844692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba281d-22eb-40e3-8397-836307b0da62 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:22.846866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 523/1021] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:22.854050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61fb6c-f3e8-4116-bf4f-1ad27407df6a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:22.854842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61fb6c-f3e8-4116-bf4f-1ad27407df6a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:22.854842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61fb6c-f3e8-4116-bf4f-1ad27407df6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:22.876025 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-aa61fb6c-f3e8-4116-bf4f-1ad27407df6a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:22.877701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aa61fb6c-f3e8-4116-bf4f-1ad27407df6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:22.921826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:23.017746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 499/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:23.314176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba281d-22eb-40e3-8397-836307b0da62 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:23.526623 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba281d-22eb-40e3-8397-836307b0da62 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:23.598838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a9a9f-7b47-443a-954a-3fde5bbfff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:23.599928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a9a9f-7b47-443a-954a-3fde5bbfff82 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:23.600453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a9a9f-7b47-443a-954a-3fde5bbfff82 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:23.606156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a9a9f-7b47-443a-954a-3fde5bbfff82 None admin] PATH_INFO: `/v3/services/image` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:23.607361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a9a9f-7b47-443a-954a-3fde5bbfff82 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:23.610071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a9a9f-7b47-443a-954a-3fde5bbfff82 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b94a9a9f-7b47-443a-954a-3fde5bbfff82 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:23.628326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:23.638345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:23.640150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 524/1023] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Sep 7 12:53:23 2022] GET /identity/v3/services/image => generated 86 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:23.644249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61fb6c-f3e8-4116-bf4f-1ad27407df6a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:23.745352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b1c00-9ee1-4734-a73b-e520a83954b0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:23.745657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b1c00-9ee1-4734-a73b-e520a83954b0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:23.745953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b1c00-9ee1-4734-a73b-e520a83954b0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:23.747338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 525/1024] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:23.753615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb1bb2-08fc-4ac6-8b4f-ab6c07740b7b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:23.753962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb1bb2-08fc-4ac6-8b4f-ab6c07740b7b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:23.754941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb1bb2-08fc-4ac6-8b4f-ab6c07740b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:23.783668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bcbb1bb2-08fc-4ac6-8b4f-ab6c07740b7b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:23.785022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bcbb1bb2-08fc-4ac6-8b4f-ab6c07740b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:23.838786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61fb6c-f3e8-4116-bf4f-1ad27407df6a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:23.864931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:23.904034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:23.904932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:23.904932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:23.905247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:23.905983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:23.906293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:23.907876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:23.914798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 500/1025] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:23.919341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 526/1026] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:23.920795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:23.964109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:24.080916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938050e-45e3-457f-930c-68c8525c3ff6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:24.152128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:24.153046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:24.153513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:24.154009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:24.154925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] RBAC: Adding query filter params (type=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:24.155352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:24.157370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:24.166090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 501/1027] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:24.174706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb1bb2-08fc-4ac6-8b4f-ab6c07740b7b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:24.333001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb1bb2-08fc-4ac6-8b4f-ab6c07740b7b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:24.380638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:24.381585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:24.381585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:24.381585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:24.383014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] RBAC: Adding query filter params (interface=public, service_id=d2817708004f426e90917c1b128f40cb, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:24.383326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:24.384796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:24.401527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 527/1028] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:53:24 2022] GET /identity/v3/endpoints?service_id=d2817708004f426e90917c1b128f40cb&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:24.403935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:24.442706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:24.510117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:24.587204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:24.592288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269b752-ec17-4106-a8d6-8503c65d1126 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:24.610361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdab21-3e80-456b-adcd-7f5c3a9e4b6d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:24.655663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:24.656659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:24.657147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:24.657615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:24.660278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:24.663562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:24.664615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc424b9-1c4e-49ac-b372-4a12f50996c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:24.665247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc424b9-1c4e-49ac-b372-4a12f50996c7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:24.665525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc424b9-1c4e-49ac-b372-4a12f50996c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:24.665931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc424b9-1c4e-49ac-b372-4a12f50996c7 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:24.666769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc424b9-1c4e-49ac-b372-4a12f50996c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:24.668174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc424b9-1c4e-49ac-b372-4a12f50996c7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:24.670187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 528/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:24.675817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:24.684183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 502/1030] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:53:24 2022] GET /identity/v3/services => generated 1957 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 12:53:24.901705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584f916-7709-4bc6-aaca-bade4847682c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:24.925218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:24.974708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462d919-af08-4b72-a185-e08a66ca2bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:24.975661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d919-af08-4b72-a185-e08a66ca2bc4 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:24.976186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d919-af08-4b72-a185-e08a66ca2bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:24.976497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d919-af08-4b72-a185-e08a66ca2bc4 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:24.977200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462d919-af08-4b72-a185-e08a66ca2bc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:24.978740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462d919-af08-4b72-a185-e08a66ca2bc4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:25.010593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 529/1031] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:53:24 2022] POST /identity/v3/users => generated 285 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:53:25.018300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b16c4-960a-4b77-a71b-b7d0b0d77354 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:25.019165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b16c4-960a-4b77-a71b-b7d0b0d77354 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:25.019351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b16c4-960a-4b77-a71b-b7d0b0d77354 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:25.020994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 503/1032] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:25.031278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20e60c-9588-4063-bfcd-19d4b3c7d7cc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:25.031752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20e60c-9588-4063-bfcd-19d4b3c7d7cc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:25.032235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20e60c-9588-4063-bfcd-19d4b3c7d7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:25.066325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1d20e60c-9588-4063-bfcd-19d4b3c7d7cc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:25.068412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1d20e60c-9588-4063-bfcd-19d4b3c7d7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:25.226135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 530/1033] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:25.615001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b16c4-960a-4b77-a71b-b7d0b0d77354 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:25.816927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b16c4-960a-4b77-a71b-b7d0b0d77354 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:25.879604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:25.880258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:25.880596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:25.881169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:25.897540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:25.902795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:25.904222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:25.905143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 504/1034] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 12:53:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:25.910475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20e60c-9588-4063-bfcd-19d4b3c7d7cc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:26.047464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:26.060182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20e60c-9588-4063-bfcd-19d4b3c7d7cc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:26.119452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:26.120515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:26.121006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:26.121481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:26.123042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:26.123573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:26.126475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:26.150123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 531/1035] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 12:53:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:26.152931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:26.189531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:26.385045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c3105-6b7e-4d7b-b8ad-f87e47aacc9e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:26.461126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:26.462223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:26.462701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:26.463133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:26.485151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:26.488636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:26.521887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:26.523802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:26.525609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:26.525609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:26.525609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:26.525609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 505/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:26.530701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:26.680970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ebf84-10ce-4527-9010-b0c57712b6ad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:26.723523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:26.724266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:26.724971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:26.725323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:26.726174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:26.726620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:26.728505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:26.751781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 532/1037] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:26.753620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:26.846064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:26.891527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8722b9-22df-4383-9ada-b53ec752fa32 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:26.939204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:26.940090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:26.940371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:26.940649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:26.951219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:26.953071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:53:26.969321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:26.971262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:53:26.973214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:26.977985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 506/1038] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:53:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:26.980853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:27.017514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:27.186418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8436500-ece2-4f65-91b1-f1d2b92230dd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:27.239761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53105b81-cd1b-4da4-80c1-5dcbd5a895dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:27.240375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53105b81-cd1b-4da4-80c1-5dcbd5a895dc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:27.241375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53105b81-cd1b-4da4-80c1-5dcbd5a895dc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:27.241662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53105b81-cd1b-4da4-80c1-5dcbd5a895dc None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:27.242454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53105b81-cd1b-4da4-80c1-5dcbd5a895dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:27.242770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53105b81-cd1b-4da4-80c1-5dcbd5a895dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:27.244251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53105b81-cd1b-4da4-80c1-5dcbd5a895dc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:27.271550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 533/1039] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:53:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:27.280153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:27.448608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de130e0-88ed-41c5-b0dc-4610179053f4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:27.511464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:27.529248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:27.532947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:27.532947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:27.532947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:27.660055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b7fc4-c2d0-4aef-9250-e84200495404 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:27.660322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b7fc4-c2d0-4aef-9250-e84200495404 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:27.660705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b7fc4-c2d0-4aef-9250-e84200495404 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:27.662364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 534/1040] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:27.668277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef1ac7-f554-4540-abcb-47448ae62077 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:27.668690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef1ac7-f554-4540-abcb-47448ae62077 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:27.668957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef1ac7-f554-4540-abcb-47448ae62077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:27.674421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:27.678596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:27.693921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-54ef1ac7-f554-4540-abcb-47448ae62077 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:27.695148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-54ef1ac7-f554-4540-abcb-47448ae62077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:27.799566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 535/1041] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:27.848260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 507/1042] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 12:53:27.850456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef1ac7-f554-4540-abcb-47448ae62077 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:27.866360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:28.051097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef1ac7-f554-4540-abcb-47448ae62077 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:28.101700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:28.102346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.102619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.102893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.116981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:28.119206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:28.138599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:28.147039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:28.154595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:28.156714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:28.156714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:28.156714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:28.156714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 536/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:28.169370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:28.195717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:28.273573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:28.337064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0372f05-6b13-46b6-ba01-644508ca4b55 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:28.342222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88888526-1c4e-4feb-9821-7e6aec666315 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:28.398870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:28.400244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.400755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.401228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] PATH_INFO: `/v3/services/image` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.402202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:28.404564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:28.408597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:28.409969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.410708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.411406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.412878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:28.413621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:28.415352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:28.420690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:28.421914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:28.423107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 508/1044] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Sep 7 12:53:28 2022] GET /identity/v3/services/image => generated 86 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:28.432770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:28.437758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 537/1045] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:28.447759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:28.574263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:28.588276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55250da9-5d17-4adb-8a0c-a930a72e9646 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:28.634709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6fa20-8b62-49ea-b904-eba9b7d79559 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:28.639806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a37ac-067c-4952-996c-c73c9ee8ca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:28.640756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a37ac-067c-4952-996c-c73c9ee8ca21 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.641039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a37ac-067c-4952-996c-c73c9ee8ca21 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.641311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a37ac-067c-4952-996c-c73c9ee8ca21 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.641995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a37ac-067c-4952-996c-c73c9ee8ca21 None admin] RBAC: Adding query filter params (name=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:28.642302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a37ac-067c-4952-996c-c73c9ee8ca21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:28.644259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a37ac-067c-4952-996c-c73c9ee8ca21 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:28.652779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 509/1046] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:28.654726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10f9d7-758d-4ac2-8410-68fe76f029ab None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.655223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10f9d7-758d-4ac2-8410-68fe76f029ab None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.655653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10f9d7-758d-4ac2-8410-68fe76f029ab None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.658179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 510/1047] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:28.659337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10f9d7-758d-4ac2-8410-68fe76f029ab None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:28.673550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:28.704992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbf519-cf99-4566-860f-38b1e989ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:28.706095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbf519-cf99-4566-860f-38b1e989ad22 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.706448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbf519-cf99-4566-860f-38b1e989ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.707231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbf519-cf99-4566-860f-38b1e989ad22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.719968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbf519-cf99-4566-860f-38b1e989ad22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:28.722691 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbf519-cf99-4566-860f-38b1e989ad22 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2dbbf519-cf99-4566-860f-38b1e989ad22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:53:28.743777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:28.745514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 538/1048] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3faa1-2ee6-4abe-a085-04b29b75ea5d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.747527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3faa1-2ee6-4abe-a085-04b29b75ea5d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.748753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3faa1-2ee6-4abe-a085-04b29b75ea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.771407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9db3faa1-2ee6-4abe-a085-04b29b75ea5d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:28.773193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9db3faa1-2ee6-4abe-a085-04b29b75ea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:28.819343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10f9d7-758d-4ac2-8410-68fe76f029ab None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:28.869119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:28.870026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:28.870459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:28.870872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:28.871897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] RBAC: Adding query filter params (type=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:28.872394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:28.875868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:28.887960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 511/1049] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:28.888220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 539/1050] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:28.888341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:28.889968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3faa1-2ee6-4abe-a085-04b29b75ea5d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:28.928072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:29.076238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85a77-6ebf-4b10-bc8e-e4857d2c3ebd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:29.090718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3faa1-2ee6-4abe-a085-04b29b75ea5d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:29.108962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:29.131436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:29.132358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:29.133036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:29.134043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:29.135173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:29.135627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:29.137948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:29.152286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:29.153283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:29.153795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:29.154222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:29.155421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:29.161871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 512/1051] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:53:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:29.162286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:29.166655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:29.195542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 540/1052] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:53:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:29.264138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:29.317562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:29.354622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78ad5e-a643-42fc-b4a6-f889095b3aa2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:29.429957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:29.431113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:29.431418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:29.431865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:29.433378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] RBAC: Adding query filter params (role.id=f9dfc647115f47a5b0728fc812c34794, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=bd45335f7a65416a8e2535c582f0624a) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:29.433562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:29.436023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:29.444626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 513/1053] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:53:29 2022] GET /identity/v3/role_assignments?user.id=bd45335f7a65416a8e2535c582f0624a&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=f9dfc647115f47a5b0728fc812c34794 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:29.445409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60156cf5-80bb-46c2-8032-4adfb87f12af None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:29.498018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:29.498685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:29.498961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:29.499230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:29.513984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:29.522241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:29.523686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:29.525368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 541/1054] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:29.530243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:29.724374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd982f7e-8e58-44b6-bc08-83a82e710735 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:29.791032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:29.792075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:29.792543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:29.793007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:29.794247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:29.794785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:29.797423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:29.821843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 514/1055] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:29.826480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:29.876898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:30.046887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769e1a5-98f5-455b-9b78-815247621091 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:30.129623 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:30.131076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:30.131273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:30.131741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] PATH_INFO: `/v3/users/912fa7258b324e1f9316a46c776a6bba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:30.153038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] RBAC: Authorizing `identity:get_user(user_id=912fa7258b324e1f9316a46c776a6bba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:30.154586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:30.156640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 542/1056] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:29 2022] GET /identity/v3/users/912fa7258b324e1f9316a46c776a6bba => generated 285 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:30.163602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:30.167147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:30.359902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c1435-0471-419c-960e-e5e4c5b773cd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:30.433156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:30.433951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:30.434232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:30.434626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:30.564684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:30.567650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:30.571838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:30.793538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 515/1057] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:53:30.804051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:30.986266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:31.025937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53622b22-bff2-4745-9e0f-0f5786ce9c1e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:31.100638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036d19c-8766-48c2-bad0-ab4ffa5b5616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:31.101286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036d19c-8766-48c2-bad0-ab4ffa5b5616 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:31.101573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036d19c-8766-48c2-bad0-ab4ffa5b5616 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:31.101889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036d19c-8766-48c2-bad0-ab4ffa5b5616 None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:31.103128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036d19c-8766-48c2-bad0-ab4ffa5b5616 None admin] RBAC: Authorizing `identity:get_project(project_id=345359a4b3db4ba688de7e944bc8e831)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:31.104620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036d19c-8766-48c2-bad0-ab4ffa5b5616 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:31.106878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 543/1058] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:53:30 2022] GET /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831 => generated 311 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:31.114732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:31.302835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473d69a-1970-49aa-97a2-bbc60974e90a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:31.304834 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:31.390060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f60e2-6ef5-4770-b0dd-c19bb8c808bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:31.391360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f60e2-6ef5-4770-b0dd-c19bb8c808bd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:31.391701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f60e2-6ef5-4770-b0dd-c19bb8c808bd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:31.391980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f60e2-6ef5-4770-b0dd-c19bb8c808bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:31.393315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f60e2-6ef5-4770-b0dd-c19bb8c808bd None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=345359a4b3db4ba688de7e944bc8e831, user.id=912fa7258b324e1f9316a46c776a6bba) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:31.393867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f60e2-6ef5-4770-b0dd-c19bb8c808bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:31.396531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f60e2-6ef5-4770-b0dd-c19bb8c808bd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:31.411400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 516/1059] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:53:31 2022] GET /identity/v3/role_assignments?user.id=912fa7258b324e1f9316a46c776a6bba&scope.project.id=345359a4b3db4ba688de7e944bc8e831&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:32.229260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9623fcf-27d0-4e88-ac7d-e38db0647b9b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:32.230130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9623fcf-27d0-4e88-ac7d-e38db0647b9b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:32.230130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9623fcf-27d0-4e88-ac7d-e38db0647b9b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:32.231953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 544/1060] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:32.238794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437d7b-781f-4552-b003-4b48e11c950e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:32.239277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437d7b-781f-4552-b003-4b48e11c950e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:32.239680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437d7b-781f-4552-b003-4b48e11c950e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:32.262376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-af437d7b-781f-4552-b003-4b48e11c950e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:32.263877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-af437d7b-781f-4552-b003-4b48e11c950e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:32.371894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 517/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:32.697398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9623fcf-27d0-4e88-ac7d-e38db0647b9b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:32.846622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9623fcf-27d0-4e88-ac7d-e38db0647b9b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:32.897944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:32.898584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:32.898891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:32.899137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:32.922784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:32.991163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:32.991964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527695-61eb-4de5-91a2-79b83d2e3628 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:32.991964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527695-61eb-4de5-91a2-79b83d2e3628 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:32.995626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527695-61eb-4de5-91a2-79b83d2e3628 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:32.995866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:33.001964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 518/1062] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:32 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:33.004295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01e0b2-c65a-42cc-95bd-88567bfb2194 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:33.004749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01e0b2-c65a-42cc-95bd-88567bfb2194 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:33.005238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01e0b2-c65a-42cc-95bd-88567bfb2194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:33.036687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ed01e0b2-c65a-42cc-95bd-88567bfb2194 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:33.038910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed01e0b2-c65a-42cc-95bd-88567bfb2194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:33.149732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 545/1063] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 12:53:33.152841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:33.172886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 519/1064] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:33.217078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:33.291105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d5d29-6ad3-4381-95c1-8110a266a584 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:33.333889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:33.335844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:33.336258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:33.336508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:33.349399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:33.351911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:33.372146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:33.374127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:33.379958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:33.379958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:33.379958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:33.379958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 546/1065] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:33.383943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01e0b2-c65a-42cc-95bd-88567bfb2194 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:33.427196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:33.528080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01e0b2-c65a-42cc-95bd-88567bfb2194 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:33.564768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:33.582524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:33.583330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:33.583799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:33.584301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:33.585111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:33.586185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:33.587602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:33.606304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 520/1066] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:33.757194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dc087-38c7-4aec-9673-79958ff8be5c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:33.759989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:33.825023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:33.825887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:33.826279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:33.826656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:33.848307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:33.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:33.860225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:33.861382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 547/1067] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 12:53:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:33.871056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:34.011532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:34.035434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fde30-79ba-4786-b038-2449e2b95139 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:34.083379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:34.084396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:34.084678 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:34.085001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:34.085654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:34.085991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:34.087471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:34.099220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 521/1068] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 12:53:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:34.107585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:34.252975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ba570-82aa-4962-b30c-67ff734ddca4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:34.308902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29390072-c167-4f7a-a82c-f90c6bfea41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:34.309830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29390072-c167-4f7a-a82c-f90c6bfea41d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:34.309993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29390072-c167-4f7a-a82c-f90c6bfea41d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:34.310211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29390072-c167-4f7a-a82c-f90c6bfea41d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:34.322975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29390072-c167-4f7a-a82c-f90c6bfea41d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:34.324496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29390072-c167-4f7a-a82c-f90c6bfea41d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-29390072-c167-4f7a-a82c-f90c6bfea41d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:34.341555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:34.348762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:34.349843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:34.349843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:34.349843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:34.349843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 548/1069] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:34.350093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:34.374954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06d90e-7ac5-4667-bcfb-90cbcec3de31 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:34.375574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06d90e-7ac5-4667-bcfb-90cbcec3de31 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:34.376017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06d90e-7ac5-4667-bcfb-90cbcec3de31 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:34.378187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 549/1070] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:34.385456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813fc31-3a5e-46d0-a35d-8c2ad1f22e1c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:34.385839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813fc31-3a5e-46d0-a35d-8c2ad1f22e1c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:34.386172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813fc31-3a5e-46d0-a35d-8c2ad1f22e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:34.413505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2813fc31-3a5e-46d0-a35d-8c2ad1f22e1c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:34.415478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2813fc31-3a5e-46d0-a35d-8c2ad1f22e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:34.431910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:34.516220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a498da-0ed9-49e4-887e-2896090a670f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:34.516465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 550/1071] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:34.577539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:34.578587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:34.579075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:34.579546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:34.580660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:34.581205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:34.583741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:34.605403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 522/1072] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:34.611702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813fc31-3a5e-46d0-a35d-8c2ad1f22e1c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:34.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813fc31-3a5e-46d0-a35d-8c2ad1f22e1c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:34.822206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:34.822863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:34.823139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:34.823409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:34.832022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:34.833504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:53:34.846990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:34.849007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:34.850950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 551/1073] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:53:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:34.853545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:34.856622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:34.861296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:34.867528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:34.996134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b70a4d-9f84-49a4-8c28-ccfeaccecd3c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:34.998948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295d944-8b33-4160-a804-23071ad71985 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.050501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.051247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.051433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.051592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.051871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.052036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.052296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.052441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.053564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:35.054795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.056781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.071065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:35.079701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:35.081808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:35.082684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 523/1074] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:35.088868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.089981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 552/1075] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:53:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:35.096168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.175129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:35.177828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:35.221941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee040ec-cd46-4f51-80d9-7fae0e60e18b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.227119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5369c-7fcc-44d8-b368-da22c5aadaf2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.270509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.271503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.271947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.272387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.273790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:35.274256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.275810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.280421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.281075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.281394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.281699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.292080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 524/1076] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:53:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:35.297178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.297919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.299609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:35.318087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:35.320351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:35.321996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:35.321996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:35.321996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:35.321996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 553/1077] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:35.329541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.440028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d53140-1b83-4997-a76e-1bbb2ba1e561 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.479063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:35.489978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.490993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.491428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.493877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] PATH_INFO: `/v3/users/912fa7258b324e1f9316a46c776a6bba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.494056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=912fa7258b324e1f9316a46c776a6bba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.497009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.500178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 525/1078] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:35 2022] GET /identity/v3/users/912fa7258b324e1f9316a46c776a6bba => generated 285 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:35.508882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.511288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea63eda-ab7c-4b32-9e44-0f709bd2efa9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.533176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:35.561530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.562184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.562576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.562721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.563346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:35.563640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.565048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.581046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 554/1079] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:35.590055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.666189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca40749-fb9f-47ff-847f-51ea67818ad3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.728099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.729071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.729540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.730035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.731705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15baedf7-23df-4985-9a52-03392a537513 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.732118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] RBAC: Authorizing `identity:get_project(project_id=345359a4b3db4ba688de7e944bc8e831)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.734733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.736712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 526/1080] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:53:35 2022] GET /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831 => generated 311 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:35.743580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.780307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.780951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.781227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.781495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.789742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.791265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:53:35.805057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:35.806884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:35.808762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 555/1081] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:35.813284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:35.852240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:35.860166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:35.911784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5848fa9-26e0-4aa5-ace1-087690030f5a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.955674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53ebb-4ad3-44cc-914c-7cfd13cb9d19 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:35.955981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:35.961327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:35.961327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:35.961327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831/users/912fa7258b324e1f9316a46c776a6bba/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:35.961327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] RBAC: Authorizing `identity:create_grant(project_id=345359a4b3db4ba688de7e944bc8e831, user_id=912fa7258b324e1f9316a46c776a6bba, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:35.962690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:35.985138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 527/1082] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:53:35 2022] PUT /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831/users/912fa7258b324e1f9316a46c776a6bba/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:53:36.003794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab7bc9-1e6c-4702-9d97-d304062b8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:36.004440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab7bc9-1e6c-4702-9d97-d304062b8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:36.004717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab7bc9-1e6c-4702-9d97-d304062b8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:36.005030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab7bc9-1e6c-4702-9d97-d304062b8cd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:36.005931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab7bc9-1e6c-4702-9d97-d304062b8cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:36.006865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab7bc9-1e6c-4702-9d97-d304062b8cd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:36.008565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab7bc9-1e6c-4702-9d97-d304062b8cd2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:36.024561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 556/1083] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:36.031452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:36.177663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236afed5-7ee3-4522-9144-ca1bc98d14ce None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:36.218136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:36.234292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecb6cb-1300-4a12-8954-5ec9994faa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:36.235229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecb6cb-1300-4a12-8954-5ec9994faa6c None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:36.235594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecb6cb-1300-4a12-8954-5ec9994faa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:36.235976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecb6cb-1300-4a12-8954-5ec9994faa6c None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/bd45335f7a65416a8e2535c582f0624a/roles/f9dfc647115f47a5b0728fc812c34794` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:36.239021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecb6cb-1300-4a12-8954-5ec9994faa6c None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=bd45335f7a65416a8e2535c582f0624a, role_id=f9dfc647115f47a5b0728fc812c34794)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:36.240959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecb6cb-1300-4a12-8954-5ec9994faa6c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:36.264979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 528/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:53:36 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/bd45335f7a65416a8e2535c582f0624a/roles/f9dfc647115f47a5b0728fc812c34794 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:53:38.611718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974edcf-170c-4f5b-8483-c089d420143f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:38.612589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974edcf-170c-4f5b-8483-c089d420143f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:38.612589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974edcf-170c-4f5b-8483-c089d420143f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:38.614251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 557/1085] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:38.620779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16e68-55b1-4521-a095-13d94b2f7f61 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:38.621530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16e68-55b1-4521-a095-13d94b2f7f61 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:38.621530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16e68-55b1-4521-a095-13d94b2f7f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:38.644892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e9a16e68-55b1-4521-a095-13d94b2f7f61 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:38.646168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e9a16e68-55b1-4521-a095-13d94b2f7f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:38.743862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 529/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:39.060321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974edcf-170c-4f5b-8483-c089d420143f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:39.175286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:39.186681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974edcf-170c-4f5b-8483-c089d420143f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:39.228137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:39.228786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:39.229028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:39.229330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:39.243866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:39.249125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:39.250984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:39.253285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 558/1087] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:53:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:39.259286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16e68-55b1-4521-a095-13d94b2f7f61 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:39.387907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16e68-55b1-4521-a095-13d94b2f7f61 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:39.413959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:39.440684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef879d8-bce8-41a0-8696-e2da48417591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:39.441312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef879d8-bce8-41a0-8696-e2da48417591 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:39.441586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef879d8-bce8-41a0-8696-e2da48417591 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:39.441874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef879d8-bce8-41a0-8696-e2da48417591 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:39.442546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef879d8-bce8-41a0-8696-e2da48417591 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:39.442846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef879d8-bce8-41a0-8696-e2da48417591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:39.445343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef879d8-bce8-41a0-8696-e2da48417591 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:39.455342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 530/1088] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:39.462055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:39.472307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938b16-1103-48d5-9572-a00eeeeaf5e5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:39.472772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938b16-1103-48d5-9572-a00eeeeaf5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:39.473224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58938b16-1103-48d5-9572-a00eeeeaf5e5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:39.476076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 531/1089] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:39.484313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f06fa-f5b8-4205-ae00-c5b9f6f40e9f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:39.484762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f06fa-f5b8-4205-ae00-c5b9f6f40e9f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:39.485343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f06fa-f5b8-4205-ae00-c5b9f6f40e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:39.513757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fe5f06fa-f5b8-4205-ae00-c5b9f6f40e9f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:39.515848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fe5f06fa-f5b8-4205-ae00-c5b9f6f40e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:39.614797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bceab-4123-495a-8de4-fede42bf67fe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:39.618856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 532/1090] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:53:39.678769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:39.679738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:39.680224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:39.680683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] PATH_INFO: `/v3/users/912fa7258b324e1f9316a46c776a6bba` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:39.682741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] RBAC: Authorizing `identity:get_user(user_id=912fa7258b324e1f9316a46c776a6bba)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:39.685329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:39.688489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 559/1091] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:39 2022] GET /identity/v3/users/912fa7258b324e1f9316a46c776a6bba => generated 285 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:39.695388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f06fa-f5b8-4205-ae00-c5b9f6f40e9f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:39.879470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f06fa-f5b8-4205-ae00-c5b9f6f40e9f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:39.895594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:39.937880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:39.938594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:39.938879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:39.939201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:40.014574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:40.055289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:40.065922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:40.069046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:40.189308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435fbd9-df92-4ef2-b921-8417ac814353 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:40.253337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:40.254371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:40.254714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:40.254988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:40.265917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 533/1092] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:53:40.272352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:40.272875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:53:40.283255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:40.285391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:40.288901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 560/1093] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 12:53:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:40.292810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:40.293710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:40.436207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081a005-963a-4da0-8c37-7fd6091e0284 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:40.437957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:40.467037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f469a9-76fe-4d6f-a385-86189881c30d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:40.495315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:40.496128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:40.496495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:40.496876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:40.498723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] RBAC: Authorizing `identity:get_project(project_id=345359a4b3db4ba688de7e944bc8e831)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:40.500960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:40.503234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 534/1094] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:53:40 2022] GET /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831 => generated 311 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:40.510825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:40.517598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:40.518256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:40.518528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:40.518791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:40.519452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:40.519749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:40.521320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:40.533221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 561/1095] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 12:53:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:40.540417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:40.628283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:40.673350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf236c-7d56-460d-9411-d8f96682abb6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:40.683039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93379c-f974-4073-ab6b-a86c2fce3eb8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:40.732840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:40.733423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:40.733694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:40.733954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:40.734631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:40.734749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=345359a4b3db4ba688de7e944bc8e831, user.id=912fa7258b324e1f9316a46c776a6bba) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:40.735001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:40.735599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:40.736014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:40.736424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:40.736513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:40.743860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 535/1096] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:53:40 2022] GET /identity/v3/role_assignments?user.id=912fa7258b324e1f9316a46c776a6bba&scope.project.id=345359a4b3db4ba688de7e944bc8e831&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:40.758173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:40.760781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:40.766078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:40.785938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:40.788168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:40.790344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:40.790344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:40.790344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:40.790344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 562/1097] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:53:40.795295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:40.946611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f3677-7ddc-45d2-8c00-663c0f041c6f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:40.996381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:40.997305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:40.997889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:40.998329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:40.999418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:40.999797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:41.002290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:41.019599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:41.025866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 536/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:41.039146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:41.167077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb48da-64b8-4a24-8ef3-0fed830a6012 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:41.207499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:41.208144 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:41.208421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:41.208703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:41.217461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:41.219099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:53:41.232686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:41.234636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:53:41.236398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 563/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 12:53:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:41.241191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:41.418318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115cb9b-d938-4168-b6a1-0113131628d0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:41.471229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:41.471935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:41.472219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:41.472499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:41.473280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:41.473588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:41.475428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:41.496970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 537/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 12:53:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:41.504159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:41.574839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:41.637102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39acda-f46d-4f32-84e0-f50e367c06f8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:41.694896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:41.695579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:41.695920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:41.696892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:41.804797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:41.807400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:41.886417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:41.943336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 564/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:53:41.951123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:42.049359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:42.108898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9822454-ee18-428a-b34d-b370e6beb074 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:42.156965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:42.157613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:42.157924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:42.158201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:42.174875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:42.178107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:53:42.205790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:42.207715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:53:42.209455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:53:42.209455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:53:42.209455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:42.209455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 538/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:42.218665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:42.360795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295c319-8138-4d0d-bbd4-501dc5690dff None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:42.411279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:42.411937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:42.412232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:42.412501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:42.413158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:42.413454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:42.414943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:42.436910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 565/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:42.446698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:42.585713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0759d9-73fb-4783-bdb9-d2d837d1e894 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:42.602434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:42.634934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:42.635587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:42.635867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:42.636142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:42.654908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:42.656400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:53:42.669246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:53:42.679771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:53:42.681939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 539/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:53:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:53:42.685342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:42.728067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:42.840528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bf67e-bd96-4530-8b03-6c144006a4ea None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:42.890207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050883f-7cf3-4e0f-80da-c065aa1ecb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:42.891216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050883f-7cf3-4e0f-80da-c065aa1ecb36 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:42.891655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050883f-7cf3-4e0f-80da-c065aa1ecb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:42.892141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050883f-7cf3-4e0f-80da-c065aa1ecb36 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:42.893268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050883f-7cf3-4e0f-80da-c065aa1ecb36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:42.893793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050883f-7cf3-4e0f-80da-c065aa1ecb36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:42.896260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050883f-7cf3-4e0f-80da-c065aa1ecb36 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:42.919282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 566/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:53:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:42.921939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:43.056630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3b01-76a4-446d-8fc0-09d9ec36a3ea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:43.106802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fca26-873c-4962-a9a6-4655cf438db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:43.107404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fca26-873c-4962-a9a6-4655cf438db7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:43.107743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fca26-873c-4962-a9a6-4655cf438db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:43.108071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fca26-873c-4962-a9a6-4655cf438db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:43.108870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fca26-873c-4962-a9a6-4655cf438db7 None admin] RBAC: Adding query filter params (role.id=f9dfc647115f47a5b0728fc812c34794, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=bd45335f7a65416a8e2535c582f0624a) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:43.109243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fca26-873c-4962-a9a6-4655cf438db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:43.110853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fca26-873c-4962-a9a6-4655cf438db7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:43.118788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 540/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:53:42 2022] GET /identity/v3/role_assignments?user.id=bd45335f7a65416a8e2535c582f0624a&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=f9dfc647115f47a5b0728fc812c34794 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:43.537294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60983b-f8c3-46b4-a1e8-74e62b8078ce None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:43.537665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60983b-f8c3-46b4-a1e8-74e62b8078ce None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:43.538373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60983b-f8c3-46b4-a1e8-74e62b8078ce None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:43.541220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 567/1107] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:43.547871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f51d-75a1-4c5b-89da-59faa57e0194 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:43.548211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f51d-75a1-4c5b-89da-59faa57e0194 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:43.548519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f51d-75a1-4c5b-89da-59faa57e0194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:43.578139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c5d4f51d-75a1-4c5b-89da-59faa57e0194 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:43.580132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c5d4f51d-75a1-4c5b-89da-59faa57e0194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:43.621267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:43.682707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 541/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:44.098507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60983b-f8c3-46b4-a1e8-74e62b8078ce None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:44.216728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:44.259619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60983b-f8c3-46b4-a1e8-74e62b8078ce None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:44.318939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5366a-5fd4-4264-ad6c-251a733d19c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:44.319978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5366a-5fd4-4264-ad6c-251a733d19c0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:44.320421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5366a-5fd4-4264-ad6c-251a733d19c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:44.320840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5366a-5fd4-4264-ad6c-251a733d19c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:44.322489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5366a-5fd4-4264-ad6c-251a733d19c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:44.324094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5366a-5fd4-4264-ad6c-251a733d19c0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:44.325855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 568/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:44.336436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4f51d-75a1-4c5b-89da-59faa57e0194 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:44.534744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4f51d-75a1-4c5b-89da-59faa57e0194 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:44.597567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45583e73-2638-408c-985e-d215547c793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:44.598680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45583e73-2638-408c-985e-d215547c793e None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:44.599185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45583e73-2638-408c-985e-d215547c793e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:44.600308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45583e73-2638-408c-985e-d215547c793e None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:44.601566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45583e73-2638-408c-985e-d215547c793e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:44.604353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45583e73-2638-408c-985e-d215547c793e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:44.637792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 542/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:53:44 2022] POST /identity/v3/users => generated 286 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:53:47.049761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda276a-2535-4128-b7d8-861035e772d9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:47.050597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda276a-2535-4128-b7d8-861035e772d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:47.050597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda276a-2535-4128-b7d8-861035e772d9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:47.052178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 569/1111] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:47.060666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28dc01-3dbd-44e6-9f55-2ca9b12beb79 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:47.061545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28dc01-3dbd-44e6-9f55-2ca9b12beb79 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:47.061766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28dc01-3dbd-44e6-9f55-2ca9b12beb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:47.084510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-af28dc01-3dbd-44e6-9f55-2ca9b12beb79 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:47.085866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-af28dc01-3dbd-44e6-9f55-2ca9b12beb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:47.110108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:47.185960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 543/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:47.491199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda276a-2535-4128-b7d8-861035e772d9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:47.629852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda276a-2535-4128-b7d8-861035e772d9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:47.672612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:47.673149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:47.673429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:47.673725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] PATH_INFO: `/v3/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:47.675269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] RBAC: Authorizing `identity:get_role(role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:47.676683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:47.678185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 570/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:47 2022] GET /identity/v3/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 223 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:47.684779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28dc01-3dbd-44e6-9f55-2ca9b12beb79 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:47.814358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28dc01-3dbd-44e6-9f55-2ca9b12beb79 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:47.882020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:47.882829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:47.883248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:47.883639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] PATH_INFO: `/v3/users/9b48fa68329c4ef89fdc6531838fdd09` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:47.908691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] RBAC: Authorizing `identity:get_user(user_id=9b48fa68329c4ef89fdc6531838fdd09)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:47.911336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:47.914428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 544/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:47 2022] GET /identity/v3/users/9b48fa68329c4ef89fdc6531838fdd09 => generated 286 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:47.923771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:47.947190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:48.061043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e5a78-6bf2-4ffa-a944-5cadf445cc7b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:48.102889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:48.103584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:48.103863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:48.104133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:48.217361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:48.220031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:48.247190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:48.354231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 571/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:53:48.361244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:48.378652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:48.505364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df61f5-8503-4c98-aff7-1d866fb34210 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:48.560920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef9281-6f07-40f9-8a24-772d19db752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:48.561592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef9281-6f07-40f9-8a24-772d19db752a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:48.561909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef9281-6f07-40f9-8a24-772d19db752a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:48.562190 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef9281-6f07-40f9-8a24-772d19db752a None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:48.563629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef9281-6f07-40f9-8a24-772d19db752a None admin] RBAC: Authorizing `identity:get_project(project_id=345359a4b3db4ba688de7e944bc8e831)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:48.565157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef9281-6f07-40f9-8a24-772d19db752a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:48.567061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 545/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:53:48 2022] GET /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831 => generated 311 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:48.574807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:48.713840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb0a94-2162-475d-98f2-5dcf1d977db1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:48.730567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:48.767063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a075f4-2e66-4281-b7a2-088ad019c4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:48.767714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a075f4-2e66-4281-b7a2-088ad019c4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:48.767963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a075f4-2e66-4281-b7a2-088ad019c4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:48.768313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a075f4-2e66-4281-b7a2-088ad019c4f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:48.769072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a075f4-2e66-4281-b7a2-088ad019c4f5 None admin] RBAC: Adding query filter params (role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=345359a4b3db4ba688de7e944bc8e831, user.id=9b48fa68329c4ef89fdc6531838fdd09) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:48.769402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a075f4-2e66-4281-b7a2-088ad019c4f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:48.770959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a075f4-2e66-4281-b7a2-088ad019c4f5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:48.777873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 572/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:53:48 2022] GET /identity/v3/role_assignments?user.id=9b48fa68329c4ef89fdc6531838fdd09&scope.project.id=345359a4b3db4ba688de7e944bc8e831&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:53:51.046467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596983-3956-4fe3-a77f-35a26f5c46e6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:51.047325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596983-3956-4fe3-a77f-35a26f5c46e6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:51.047325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596983-3956-4fe3-a77f-35a26f5c46e6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:51.049804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 546/1118] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:51.057023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e57b2-2f68-4a88-8fc1-98eed2d86f9f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:51.057590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e57b2-2f68-4a88-8fc1-98eed2d86f9f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:51.057590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e57b2-2f68-4a88-8fc1-98eed2d86f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:51.081473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dc6e57b2-2f68-4a88-8fc1-98eed2d86f9f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:51.082842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc6e57b2-2f68-4a88-8fc1-98eed2d86f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:51.206129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 573/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:51.716082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5596983-3956-4fe3-a77f-35a26f5c46e6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:51.839077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:51.885926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5596983-3956-4fe3-a77f-35a26f5c46e6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:51.927474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:51.928180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:51.928463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:51.928739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] PATH_INFO: `/v3/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:51.930332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] RBAC: Authorizing `identity:get_role(role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:51.931733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:51.933138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 547/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:51 2022] GET /identity/v3/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 223 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:51.939699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e57b2-2f68-4a88-8fc1-98eed2d86f9f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:52.076495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e57b2-2f68-4a88-8fc1-98eed2d86f9f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:52.085985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:52.120966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:52.121764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:52.122049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:52.122325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] PATH_INFO: `/v3/users/9b48fa68329c4ef89fdc6531838fdd09` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:52.123569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] RBAC: Authorizing `identity:get_user(user_id=9b48fa68329c4ef89fdc6531838fdd09)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:52.125063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:52.127181 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 574/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:51 2022] GET /identity/v3/users/9b48fa68329c4ef89fdc6531838fdd09 => generated 286 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:52.134471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:52.292675 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac11224-7c3d-49d6-955b-d3f4fe6b361f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:52.346533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56eaa-4a4f-4932-810f-2dd97b030b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:52.347061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56eaa-4a4f-4932-810f-2dd97b030b4d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:52.347335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56eaa-4a4f-4932-810f-2dd97b030b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:52.347707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56eaa-4a4f-4932-810f-2dd97b030b4d None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:52.348810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56eaa-4a4f-4932-810f-2dd97b030b4d None admin] RBAC: Authorizing `identity:get_project(project_id=345359a4b3db4ba688de7e944bc8e831)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:52.350352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56eaa-4a4f-4932-810f-2dd97b030b4d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:52.352067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 548/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:53:52 2022] GET /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831 => generated 311 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:52.359373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:52.483115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf1b1d-17af-41d9-a6e6-a5471b80e8af None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:52.525320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd12bf-6099-48e9-b852-4235f1fe5439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:52.526324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd12bf-6099-48e9-b852-4235f1fe5439 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:52.526721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd12bf-6099-48e9-b852-4235f1fe5439 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:52.527128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd12bf-6099-48e9-b852-4235f1fe5439 None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831/users/9b48fa68329c4ef89fdc6531838fdd09/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:52.530011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd12bf-6099-48e9-b852-4235f1fe5439 None admin] RBAC: Authorizing `identity:create_grant(project_id=345359a4b3db4ba688de7e944bc8e831, user_id=9b48fa68329c4ef89fdc6531838fdd09, role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:52.531436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd12bf-6099-48e9-b852-4235f1fe5439 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:52.547213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 575/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:53:52 2022] PUT /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831/users/9b48fa68329c4ef89fdc6531838fdd09/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:53:54.942472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46b8d-c65c-4549-9b4c-a60b613dffb4 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:54.943417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46b8d-c65c-4549-9b4c-a60b613dffb4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:54.943417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46b8d-c65c-4549-9b4c-a60b613dffb4 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:54.945195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 549/1124] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:53:54.956174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16be6b-379f-4c24-bc33-5996736d0096 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:54.956174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16be6b-379f-4c24-bc33-5996736d0096 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:54.956174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16be6b-379f-4c24-bc33-5996736d0096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:54.977634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8b16be6b-379f-4c24-bc33-5996736d0096 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:54.978975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8b16be6b-379f-4c24-bc33-5996736d0096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:55.014517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:55.066251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 576/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:55.366040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46b8d-c65c-4549-9b4c-a60b613dffb4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:55.386869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:55.515290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46b8d-c65c-4549-9b4c-a60b613dffb4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:55.567815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:55.568677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:55.569155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:55.569565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] PATH_INFO: `/v3/roles/672f0ed1f4d94bb9945cbcdf4992d48e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:55.571083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] RBAC: Authorizing `identity:get_role(role_id=672f0ed1f4d94bb9945cbcdf4992d48e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:55.572473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:55.573867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 550/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:55 2022] GET /identity/v3/roles/672f0ed1f4d94bb9945cbcdf4992d48e => generated 223 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:55.580867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16be6b-379f-4c24-bc33-5996736d0096 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:55.710963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16be6b-379f-4c24-bc33-5996736d0096 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:55.752482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:55.753193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:55.753518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:55.753879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] PATH_INFO: `/v3/users/9b48fa68329c4ef89fdc6531838fdd09` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:55.755173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] RBAC: Authorizing `identity:get_user(user_id=9b48fa68329c4ef89fdc6531838fdd09)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:55.757440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:55.759208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 577/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:53:55 2022] GET /identity/v3/users/9b48fa68329c4ef89fdc6531838fdd09 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:55.766100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:55.930190 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:55.965148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea745c15-7030-430d-94d5-3198e3d238d4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:56.026310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:56.027389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:56.027729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:56.028193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:56.123510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:56.126332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:56.259202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:56.286456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 551/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:53:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:53:56.296568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:56.349559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:56.458610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29a67-e866-468f-91ef-b0d936f388f0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:56.506981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18bf9a-0285-4b55-a512-125484f29057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:56.507792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18bf9a-0285-4b55-a512-125484f29057 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:56.508165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18bf9a-0285-4b55-a512-125484f29057 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:56.508541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18bf9a-0285-4b55-a512-125484f29057 None admin] PATH_INFO: `/v3/projects/345359a4b3db4ba688de7e944bc8e831` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:56.510254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18bf9a-0285-4b55-a512-125484f29057 None admin] RBAC: Authorizing `identity:get_project(project_id=345359a4b3db4ba688de7e944bc8e831)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:56.512815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18bf9a-0285-4b55-a512-125484f29057 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:56.514690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 578/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:53:56 2022] GET /identity/v3/projects/345359a4b3db4ba688de7e944bc8e831 => generated 311 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:56.521770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:56.652822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4f63e-2849-4ba6-80f6-9724a1d7e2c0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:53:56.690777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d29f-cdae-4635-b18a-90044e7a2f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:53:56.691440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d29f-cdae-4635-b18a-90044e7a2f6b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:56.691762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d29f-cdae-4635-b18a-90044e7a2f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:56.692120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d29f-cdae-4635-b18a-90044e7a2f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:56.693456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003d29f-cdae-4635-b18a-90044e7a2f6b None admin] RBAC: Adding query filter params (role.id=672f0ed1f4d94bb9945cbcdf4992d48e, scope.project.id=345359a4b3db4ba688de7e944bc8e831, user.id=9b48fa68329c4ef89fdc6531838fdd09) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:53:56.694518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003d29f-cdae-4635-b18a-90044e7a2f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:53:56.696621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003d29f-cdae-4635-b18a-90044e7a2f6b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:53:56.704426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 552/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:53:56 2022] GET /identity/v3/role_assignments?user.id=9b48fa68329c4ef89fdc6531838fdd09&scope.project.id=345359a4b3db4ba688de7e944bc8e831&role.id=672f0ed1f4d94bb9945cbcdf4992d48e => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:53:59.321334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a2079-4a8b-42f3-b2ea-8773bdf363b7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:59.322004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a2079-4a8b-42f3-b2ea-8773bdf363b7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:59.322219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a2079-4a8b-42f3-b2ea-8773bdf363b7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:59.324370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 579/1131] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:53:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:53:59.332422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358954-25cf-41e6-8a3c-a9c7690eced4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:53:59.333111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358954-25cf-41e6-8a3c-a9c7690eced4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:53:59.333111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358954-25cf-41e6-8a3c-a9c7690eced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:53:59.358405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2b358954-25cf-41e6-8a3c-a9c7690eced4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:53:59.359684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b358954-25cf-41e6-8a3c-a9c7690eced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:53:59.419453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:59.455985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 553/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:53:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:53:59.804821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a2079-4a8b-42f3-b2ea-8773bdf363b7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:53:59.981028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:53:59.987938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a2079-4a8b-42f3-b2ea-8773bdf363b7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:00.045009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5828f7-1504-4c42-b626-2dc7cbe75c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:00.045750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5828f7-1504-4c42-b626-2dc7cbe75c1e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:00.046092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5828f7-1504-4c42-b626-2dc7cbe75c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:00.046414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5828f7-1504-4c42-b626-2dc7cbe75c1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:00.047770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5828f7-1504-4c42-b626-2dc7cbe75c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:00.049485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5828f7-1504-4c42-b626-2dc7cbe75c1e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:00.051592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 580/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:53:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:00.059563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358954-25cf-41e6-8a3c-a9c7690eced4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:00.201598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358954-25cf-41e6-8a3c-a9c7690eced4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:00.257457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d891f6-84f3-4881-8edc-b9916c26bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:00.258210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d891f6-84f3-4881-8edc-b9916c26bf5f None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:00.258623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d891f6-84f3-4881-8edc-b9916c26bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:00.258941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d891f6-84f3-4881-8edc-b9916c26bf5f None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:00.259657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d891f6-84f3-4881-8edc-b9916c26bf5f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:00.261278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d891f6-84f3-4881-8edc-b9916c26bf5f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:00.294399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 554/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:54:00 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:54:02.719399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c8934-c601-4661-8e16-c2c182481719 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:02.720268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c8934-c601-4661-8e16-c2c182481719 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:02.720268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c8934-c601-4661-8e16-c2c182481719 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:02.721735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 581/1135] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:02.732226 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429aa42-199b-4733-a3fe-da07881ac26e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:02.732791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429aa42-199b-4733-a3fe-da07881ac26e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:02.733126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429aa42-199b-4733-a3fe-da07881ac26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:02.764368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3429aa42-199b-4733-a3fe-da07881ac26e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:02.766376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3429aa42-199b-4733-a3fe-da07881ac26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:02.805685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:02.881075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 555/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:03.161867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c8934-c601-4661-8e16-c2c182481719 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:03.297784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c8934-c601-4661-8e16-c2c182481719 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:03.339848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f4a28-b5d8-4aed-8618-29df94da842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:03.340504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f4a28-b5d8-4aed-8618-29df94da842b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:03.340805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f4a28-b5d8-4aed-8618-29df94da842b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:03.341102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f4a28-b5d8-4aed-8618-29df94da842b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:03.343662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f4a28-b5d8-4aed-8618-29df94da842b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:03.343807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f4a28-b5d8-4aed-8618-29df94da842b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:03.345414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 582/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:54:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:03.352653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429aa42-199b-4733-a3fe-da07881ac26e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:03.512819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429aa42-199b-4733-a3fe-da07881ac26e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:03.560811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044a50a-9022-433b-83ed-e4320662c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:03.561451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044a50a-9022-433b-83ed-e4320662c2a1 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:03.561767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044a50a-9022-433b-83ed-e4320662c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:03.562043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044a50a-9022-433b-83ed-e4320662c2a1 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:03.562733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044a50a-9022-433b-83ed-e4320662c2a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:03.564215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044a50a-9022-433b-83ed-e4320662c2a1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:03.590948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 556/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:54:03 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:54:05.953719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ae2a4-10c9-4de4-8e76-715701b214dc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:05.954293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ae2a4-10c9-4de4-8e76-715701b214dc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:05.954293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ae2a4-10c9-4de4-8e76-715701b214dc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:05.955846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 583/1139] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:05.962685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee13f6-fb75-4c45-a179-46c3aeb24654 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:05.963213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee13f6-fb75-4c45-a179-46c3aeb24654 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:05.963213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee13f6-fb75-4c45-a179-46c3aeb24654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:05.983981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0aee13f6-fb75-4c45-a179-46c3aeb24654 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:05.985209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0aee13f6-fb75-4c45-a179-46c3aeb24654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:05.995135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:06.074016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 557/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:06.353512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ae2a4-10c9-4de4-8e76-715701b214dc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:06.395180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:06.470082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ae2a4-10c9-4de4-8e76-715701b214dc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:06.510461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:06.511108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:06.511387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:06.511665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:06.523256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:54:06.528164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:54:06.529474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:54:06.530447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 584/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:54:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:54:06.535943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee13f6-fb75-4c45-a179-46c3aeb24654 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:06.656649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee13f6-fb75-4c45-a179-46c3aeb24654 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:06.699635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:06.700539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:06.700987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:06.701324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:06.702342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:06.702715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:06.704686 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:06.714651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 558/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:54:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:06.721303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:06.852031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:06.861086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8b00-817e-4ed1-9ae6-7c4058a3d152 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:06.911958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:06.913107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:06.913556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:06.913932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] PATH_INFO: `/v3/users/379be714f5444e1aadd9aee9dec4d25c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:06.930912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=379be714f5444e1aadd9aee9dec4d25c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:06.932628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:06.934708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 585/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:54:06 2022] GET /identity/v3/users/379be714f5444e1aadd9aee9dec4d25c => generated 286 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:06.940830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:06.945197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:07.088504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cebc3-1a8e-42db-b76a-8067892fa5ac None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:07.128953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:07.129641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:07.130010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:07.130325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:07.216438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:07.218020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:07.222537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:07.389444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 559/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:54:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:54:07.396565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:07.543633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5a94-b973-4306-9cae-a82eebe12bf0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:07.580979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eaeaa-786a-482f-9d7d-9157414d51d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:07.581619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eaeaa-786a-482f-9d7d-9157414d51d3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:07.581927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eaeaa-786a-482f-9d7d-9157414d51d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:07.582194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eaeaa-786a-482f-9d7d-9157414d51d3 None admin] PATH_INFO: `/v3/projects/0f39ee29d59e452c9c90af1f3241af7f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:07.583462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eaeaa-786a-482f-9d7d-9157414d51d3 None admin] RBAC: Authorizing `identity:get_project(project_id=0f39ee29d59e452c9c90af1f3241af7f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:07.586003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eaeaa-786a-482f-9d7d-9157414d51d3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:07.588525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 586/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:54:07 2022] GET /identity/v3/projects/0f39ee29d59e452c9c90af1f3241af7f => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:07.595603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:07.728562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7cd6-fb86-404a-a33c-0539de1b7a7e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:07.732754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:07.773141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7caa42-bddc-4aad-b3fa-45d8ea1f6f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:07.773887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7caa42-bddc-4aad-b3fa-45d8ea1f6f40 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:07.774234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7caa42-bddc-4aad-b3fa-45d8ea1f6f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:07.774558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7caa42-bddc-4aad-b3fa-45d8ea1f6f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:07.775349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7caa42-bddc-4aad-b3fa-45d8ea1f6f40 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0f39ee29d59e452c9c90af1f3241af7f, user.id=379be714f5444e1aadd9aee9dec4d25c) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:07.775702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7caa42-bddc-4aad-b3fa-45d8ea1f6f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:07.777233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7caa42-bddc-4aad-b3fa-45d8ea1f6f40 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:07.783760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 560/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:54:07 2022] GET /identity/v3/role_assignments?user.id=379be714f5444e1aadd9aee9dec4d25c&scope.project.id=0f39ee29d59e452c9c90af1f3241af7f&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:10.359148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487456b-bc8d-40b4-9310-09925f562f32 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:10.359756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487456b-bc8d-40b4-9310-09925f562f32 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:10.359860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487456b-bc8d-40b4-9310-09925f562f32 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:10.361765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 587/1147] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:10.369765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d2c6d-7ba0-42d7-a147-793fbd78d223 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:10.370303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d2c6d-7ba0-42d7-a147-793fbd78d223 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:10.370417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d2c6d-7ba0-42d7-a147-793fbd78d223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:10.391369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ba1d2c6d-7ba0-42d7-a147-793fbd78d223 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:10.392608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ba1d2c6d-7ba0-42d7-a147-793fbd78d223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:10.479196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 561/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:10.764048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487456b-bc8d-40b4-9310-09925f562f32 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:10.796357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:10.914239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487456b-bc8d-40b4-9310-09925f562f32 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:10.971845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:10.972701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:10.973090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:10.973466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:10.988317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:54:10.993422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:54:10.995529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:54:10.997132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 588/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:54:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:54:11.002241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d2c6d-7ba0-42d7-a147-793fbd78d223 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:11.165291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:11.173522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d2c6d-7ba0-42d7-a147-793fbd78d223 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:11.223725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:11.224372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:11.224651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:11.225068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:11.225807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:11.226223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:11.227700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:11.237708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 562/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:54:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:11.247473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:11.361544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:11.387087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5194-5c09-4419-8c92-9ad573a50bcc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:11.436564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:11.437230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:11.437513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:11.437968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] PATH_INFO: `/v3/users/379be714f5444e1aadd9aee9dec4d25c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:11.439335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] RBAC: Authorizing `identity:get_user(user_id=379be714f5444e1aadd9aee9dec4d25c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:11.440812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:11.442668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 589/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:54:11 2022] GET /identity/v3/users/379be714f5444e1aadd9aee9dec4d25c => generated 286 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:11.450286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:11.592019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130bdf5-4641-468b-943c-3587bf52ddb3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:11.647859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e10d29-231f-46aa-ac0a-c366bd923f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:11.648891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e10d29-231f-46aa-ac0a-c366bd923f05 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:11.649337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e10d29-231f-46aa-ac0a-c366bd923f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:11.649807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e10d29-231f-46aa-ac0a-c366bd923f05 None admin] PATH_INFO: `/v3/projects/0f39ee29d59e452c9c90af1f3241af7f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:11.651869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e10d29-231f-46aa-ac0a-c366bd923f05 None admin] RBAC: Authorizing `identity:get_project(project_id=0f39ee29d59e452c9c90af1f3241af7f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:11.654098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e10d29-231f-46aa-ac0a-c366bd923f05 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:11.655964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 563/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:54:11 2022] GET /identity/v3/projects/0f39ee29d59e452c9c90af1f3241af7f => generated 311 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:11.663413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:11.789753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10e973-0e94-480b-a842-06827d89ab66 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:11.845372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e048924-e27e-4a19-bb93-07b4cd60322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:11.846413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e048924-e27e-4a19-bb93-07b4cd60322f None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:11.846844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e048924-e27e-4a19-bb93-07b4cd60322f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:11.847261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e048924-e27e-4a19-bb93-07b4cd60322f None admin] PATH_INFO: `/v3/projects/0f39ee29d59e452c9c90af1f3241af7f/users/379be714f5444e1aadd9aee9dec4d25c/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:11.849360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e048924-e27e-4a19-bb93-07b4cd60322f None admin] RBAC: Authorizing `identity:create_grant(project_id=0f39ee29d59e452c9c90af1f3241af7f, user_id=379be714f5444e1aadd9aee9dec4d25c, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:11.850964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e048924-e27e-4a19-bb93-07b4cd60322f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:11.872239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 590/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:54:11 2022] PUT /identity/v3/projects/0f39ee29d59e452c9c90af1f3241af7f/users/379be714f5444e1aadd9aee9dec4d25c/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:54:14.258201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb847b-fa96-4119-a308-4b0a8edcc7cc None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:14.259058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb847b-fa96-4119-a308-4b0a8edcc7cc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:14.259215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb847b-fa96-4119-a308-4b0a8edcc7cc None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:14.261466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 564/1154] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:14.268180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eee96-f26e-4c1e-9417-5c8fbfe6abc2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:14.268807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eee96-f26e-4c1e-9417-5c8fbfe6abc2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:14.268807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eee96-f26e-4c1e-9417-5c8fbfe6abc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:14.291219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a49eee96-f26e-4c1e-9417-5c8fbfe6abc2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:14.292493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a49eee96-f26e-4c1e-9417-5c8fbfe6abc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:14.296072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:14.394718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 591/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:14.910949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb847b-fa96-4119-a308-4b0a8edcc7cc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:14.967348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:15.097918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb847b-fa96-4119-a308-4b0a8edcc7cc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:15.155048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:15.155702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:15.155983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:15.156253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:15.177597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:54:15.183156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:54:15.184965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:54:15.185929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 565/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:54:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:54:15.191215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eee96-f26e-4c1e-9417-5c8fbfe6abc2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:15.332316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eee96-f26e-4c1e-9417-5c8fbfe6abc2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:15.379179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:15.379811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:15.380117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:15.380360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:15.381135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:15.382248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:15.383668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:15.393305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 592/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:54:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:15.400534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:15.539861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:15.572241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcab666-49b4-4412-bb5a-ff3fc001b945 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:15.635652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:15.636309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:15.636586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:15.636853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] PATH_INFO: `/v3/users/379be714f5444e1aadd9aee9dec4d25c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:15.638427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] RBAC: Authorizing `identity:get_user(user_id=379be714f5444e1aadd9aee9dec4d25c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:15.639910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:15.641596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 566/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:54:15 2022] GET /identity/v3/users/379be714f5444e1aadd9aee9dec4d25c => generated 286 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:54:15.648635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:15.653126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:15.784937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851768f4-735b-49f8-8101-73c9f9b59069 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:15.825302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:15.825836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:15.826227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:15.826585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:15.915049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:15.917902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:15.921556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:16.053115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 593/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:54:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:54:16.060899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:16.192353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d079-f631-46d3-8b48-0c59e63e847c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:16.234054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e858c-c455-46d9-a94f-9fa06e12cdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:16.234700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e858c-c455-46d9-a94f-9fa06e12cdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:16.234977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e858c-c455-46d9-a94f-9fa06e12cdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:16.235245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e858c-c455-46d9-a94f-9fa06e12cdd2 None admin] PATH_INFO: `/v3/projects/0f39ee29d59e452c9c90af1f3241af7f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:16.236409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e858c-c455-46d9-a94f-9fa06e12cdd2 None admin] RBAC: Authorizing `identity:get_project(project_id=0f39ee29d59e452c9c90af1f3241af7f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:16.237894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e858c-c455-46d9-a94f-9fa06e12cdd2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:16.239620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 567/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:54:16 2022] GET /identity/v3/projects/0f39ee29d59e452c9c90af1f3241af7f => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:54:16.246401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:16.404142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb15ac-dab9-464d-9e0c-1c054abec2e2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:16.405911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:16.450507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04187b7e-847f-48a4-b191-ffc833d091b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:16.451043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187b7e-847f-48a4-b191-ffc833d091b8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:16.451318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187b7e-847f-48a4-b191-ffc833d091b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:16.451590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187b7e-847f-48a4-b191-ffc833d091b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:16.452325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04187b7e-847f-48a4-b191-ffc833d091b8 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0f39ee29d59e452c9c90af1f3241af7f, user.id=379be714f5444e1aadd9aee9dec4d25c) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:16.452645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04187b7e-847f-48a4-b191-ffc833d091b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:16.454899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04187b7e-847f-48a4-b191-ffc833d091b8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:16.462130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 594/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:54:16 2022] GET /identity/v3/role_assignments?user.id=379be714f5444e1aadd9aee9dec4d25c&scope.project.id=0f39ee29d59e452c9c90af1f3241af7f&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:18.761016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44242dca-4bec-4227-823a-b2f7629ef312 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:18.761605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44242dca-4bec-4227-823a-b2f7629ef312 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:18.761605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44242dca-4bec-4227-823a-b2f7629ef312 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:18.763132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 568/1162] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:18.769131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362881d2-3ce7-4c2a-9f09-2f4085a627a5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:18.769773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362881d2-3ce7-4c2a-9f09-2f4085a627a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:18.769773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362881d2-3ce7-4c2a-9f09-2f4085a627a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:18.789817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-362881d2-3ce7-4c2a-9f09-2f4085a627a5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:18.791069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-362881d2-3ce7-4c2a-9f09-2f4085a627a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:18.871477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 595/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:19.151955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44242dca-4bec-4227-823a-b2f7629ef312 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:19.189463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:19.283807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44242dca-4bec-4227-823a-b2f7629ef312 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:19.332178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:19.332942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:19.333149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:19.333568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:19.422743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:19.424112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:19.463019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:19.581768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 569/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:54:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:54:19.589908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362881d2-3ce7-4c2a-9f09-2f4085a627a5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:19.728504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:19.736407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362881d2-3ce7-4c2a-9f09-2f4085a627a5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:19.776264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31540f-51a1-4722-9c42-b4648b48b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:19.777029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31540f-51a1-4722-9c42-b4648b48b7ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:19.777278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31540f-51a1-4722-9c42-b4648b48b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:19.777617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31540f-51a1-4722-9c42-b4648b48b7ba None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:19.785830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31540f-51a1-4722-9c42-b4648b48b7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:19.787388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31540f-51a1-4722-9c42-b4648b48b7ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-dc31540f-51a1-4722-9c42-b4648b48b7ba None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:54:19.800494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:54:19.801898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:54:19.802836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 596/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:54:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:54:19.808530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:19.939201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7043942-a873-484b-bee4-740113dd03bd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:19.987754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:20.010870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fc74a-40da-4b56-ba95-849324daaa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:20.011985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc74a-40da-4b56-ba95-849324daaa2e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:20.012471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc74a-40da-4b56-ba95-849324daaa2e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:20.012946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc74a-40da-4b56-ba95-849324daaa2e None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:20.014210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fc74a-40da-4b56-ba95-849324daaa2e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:20.014757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fc74a-40da-4b56-ba95-849324daaa2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:20.017455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fc74a-40da-4b56-ba95-849324daaa2e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:20.032197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 570/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 12:54:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:22.712602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef2896-ab58-478d-97b4-3752f58dcf06 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:22.713226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef2896-ab58-478d-97b4-3752f58dcf06 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:22.713226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef2896-ab58-478d-97b4-3752f58dcf06 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:22.714554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 597/1167] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:54:22.720855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812d892-5601-4022-b94c-374f016a72e5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:22.721711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812d892-5601-4022-b94c-374f016a72e5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:22.721711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812d892-5601-4022-b94c-374f016a72e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:22.744768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6812d892-5601-4022-b94c-374f016a72e5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:22.746060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6812d892-5601-4022-b94c-374f016a72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:22.830963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 571/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:54:23.281614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef2896-ab58-478d-97b4-3752f58dcf06 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:23.422384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef2896-ab58-478d-97b4-3752f58dcf06 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:23.463030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cb89b-89e3-43e7-a915-f4f440a2cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:23.463644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cb89b-89e3-43e7-a915-f4f440a2cf8e None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:23.463924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cb89b-89e3-43e7-a915-f4f440a2cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:23.464194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cb89b-89e3-43e7-a915-f4f440a2cf8e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:23.464885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696cb89b-89e3-43e7-a915-f4f440a2cf8e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:23.466334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696cb89b-89e3-43e7-a915-f4f440a2cf8e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:23.487687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 598/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Wed Sep 7 12:54:23 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:54:25.943614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783bb18-d84b-4ed6-93fe-7a15bac9d87b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:25.944826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783bb18-d84b-4ed6-93fe-7a15bac9d87b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:25.944826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783bb18-d84b-4ed6-93fe-7a15bac9d87b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:25.946436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 572/1170] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:25.952724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eecdb-547a-400b-96d8-3ecab95b6123 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:25.953520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eecdb-547a-400b-96d8-3ecab95b6123 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:25.953842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eecdb-547a-400b-96d8-3ecab95b6123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:25.978324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-df5eecdb-547a-400b-96d8-3ecab95b6123 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:25.979850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-df5eecdb-547a-400b-96d8-3ecab95b6123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:25.995261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:26.081814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 599/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:54:26.358043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783bb18-d84b-4ed6-93fe-7a15bac9d87b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:26.498024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783bb18-d84b-4ed6-93fe-7a15bac9d87b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:26.519478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:26.542070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2584fe-36d3-44b0-bb8c-ce371d2d99ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:26.542788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2584fe-36d3-44b0-bb8c-ce371d2d99ee None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:26.543110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2584fe-36d3-44b0-bb8c-ce371d2d99ee None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:26.543429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2584fe-36d3-44b0-bb8c-ce371d2d99ee None admin] PATH_INFO: `/v3/domains/32f58f98eb2e4b97baf28abed5354f38` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:26.558135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2584fe-36d3-44b0-bb8c-ce371d2d99ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=32f58f98eb2e4b97baf28abed5354f38)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:26.560748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2584fe-36d3-44b0-bb8c-ce371d2d99ee None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:26.564070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 573/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Sep 7 12:54:26 2022] GET /identity/v3/domains/32f58f98eb2e4b97baf28abed5354f38 => generated 267 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:26.572013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eecdb-547a-400b-96d8-3ecab95b6123 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:26.735499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eecdb-547a-400b-96d8-3ecab95b6123 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:26.783416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f34f2-d558-4717-845b-ada256e9c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:26.784172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f34f2-d558-4717-845b-ada256e9c6e8 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:26.784558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f34f2-d558-4717-845b-ada256e9c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:26.784919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f34f2-d558-4717-845b-ada256e9c6e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:26.785693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f34f2-d558-4717-845b-ada256e9c6e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:26.787298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f34f2-d558-4717-845b-ada256e9c6e8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:26.811933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:26.824923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 600/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:54:26 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:54:29.320016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1586bf-e3dd-4493-ba2f-6d9d378fdf38 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:29.320670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1586bf-e3dd-4493-ba2f-6d9d378fdf38 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:29.320890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1586bf-e3dd-4493-ba2f-6d9d378fdf38 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:29.323600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 574/1174] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:29.334255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852994da-c555-4ee0-b9d5-1d45aedec993 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:29.335154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852994da-c555-4ee0-b9d5-1d45aedec993 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:29.335154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852994da-c555-4ee0-b9d5-1d45aedec993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:29.371224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-852994da-c555-4ee0-b9d5-1d45aedec993 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:29.373342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-852994da-c555-4ee0-b9d5-1d45aedec993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:29.474835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 601/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:29.750191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1586bf-e3dd-4493-ba2f-6d9d378fdf38 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:29.877465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1586bf-e3dd-4493-ba2f-6d9d378fdf38 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:29.930577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4269b-bb31-44bf-8551-69f3000607e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:29.931457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4269b-bb31-44bf-8551-69f3000607e7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:29.931951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4269b-bb31-44bf-8551-69f3000607e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:29.932460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4269b-bb31-44bf-8551-69f3000607e7 None admin] PATH_INFO: `/v3/domains/32f58f98eb2e4b97baf28abed5354f38` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:29.934413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4269b-bb31-44bf-8551-69f3000607e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=32f58f98eb2e4b97baf28abed5354f38)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:29.936908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4269b-bb31-44bf-8551-69f3000607e7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:29.939674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 575/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Sep 7 12:54:29 2022] GET /identity/v3/domains/32f58f98eb2e4b97baf28abed5354f38 => generated 267 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:29.947024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852994da-c555-4ee0-b9d5-1d45aedec993 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:30.117203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852994da-c555-4ee0-b9d5-1d45aedec993 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:30.155433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:30.165335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49646c7-caa3-4384-a6df-b2203c77c4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:30.165989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49646c7-caa3-4384-a6df-b2203c77c4f6 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:30.166270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49646c7-caa3-4384-a6df-b2203c77c4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:30.166537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49646c7-caa3-4384-a6df-b2203c77c4f6 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:30.167201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49646c7-caa3-4384-a6df-b2203c77c4f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:30.168675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49646c7-caa3-4384-a6df-b2203c77c4f6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:30.196826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 602/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:54:29 2022] POST /identity/v3/users => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:54:32.805193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03211-51f1-42ff-b4ba-165863f035bf None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:32.805858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03211-51f1-42ff-b4ba-165863f035bf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:32.806376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03211-51f1-42ff-b4ba-165863f035bf None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:32.808640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 576/1178] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:32.817415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93372f41-632f-4f97-b909-c9c59ee977e0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:32.818370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93372f41-632f-4f97-b909-c9c59ee977e0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:32.818370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93372f41-632f-4f97-b909-c9c59ee977e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:32.841683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-93372f41-632f-4f97-b909-c9c59ee977e0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:32.843502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-93372f41-632f-4f97-b909-c9c59ee977e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:32.930929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 603/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:54:33.217721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf03211-51f1-42ff-b4ba-165863f035bf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:33.291717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:33.369102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf03211-51f1-42ff-b4ba-165863f035bf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:33.424797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:33.425436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:33.425740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:33.426007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:33.443996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:54:33.452147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:54:33.454086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:54:33.455899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 577/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:54:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:54:33.460063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93372f41-632f-4f97-b909-c9c59ee977e0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:33.597889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93372f41-632f-4f97-b909-c9c59ee977e0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:33.622106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:33.650239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:33.650879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:33.651159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:33.651435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:33.652145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:33.652453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:33.653952 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:33.664507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 604/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:54:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:33.670830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:33.851130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac75b6-8c20-4f06-a2df-08eb715630ad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:33.859759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:33.899044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:33.899820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:33.900217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:33.900574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] PATH_INFO: `/v3/users/c5b5e18d6a8140dd843d4b21e38a9e8b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:33.917786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] RBAC: Authorizing `identity:get_user(user_id=c5b5e18d6a8140dd843d4b21e38a9e8b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:33.919253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:33.920959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 578/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:54:33 2022] GET /identity/v3/users/c5b5e18d6a8140dd843d4b21e38a9e8b => generated 311 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:33.927490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:34.051943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e417c-353b-4c48-894a-352a7c2aefd3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:34.091239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:34.091858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:34.092135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:34.092403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:34.150719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:34.180731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:34.182399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:34.310311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 605/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:54:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:54:34.317256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:34.458502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc381679-941d-45f6-9d6b-23ac771cad7e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:34.499929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80318bf1-eef6-4d55-9b15-b1d96a1c24fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:34.500731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80318bf1-eef6-4d55-9b15-b1d96a1c24fa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:34.501088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80318bf1-eef6-4d55-9b15-b1d96a1c24fa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:34.501447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80318bf1-eef6-4d55-9b15-b1d96a1c24fa None admin] PATH_INFO: `/v3/projects/3b75c12a00304f15b7b175a19b8c096f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:34.503395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80318bf1-eef6-4d55-9b15-b1d96a1c24fa None admin] RBAC: Authorizing `identity:get_project(project_id=3b75c12a00304f15b7b175a19b8c096f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:34.504928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80318bf1-eef6-4d55-9b15-b1d96a1c24fa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:34.507069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 579/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:54:34 2022] GET /identity/v3/projects/3b75c12a00304f15b7b175a19b8c096f => generated 361 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:34.516745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:34.613204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:34.643807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf692c4-e0fb-4cf5-b51e-f62033f910e9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:34.689017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f319cc0-02e1-42dd-9487-a32e1a97caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:34.689832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f319cc0-02e1-42dd-9487-a32e1a97caa5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:34.690208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f319cc0-02e1-42dd-9487-a32e1a97caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:34.690567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f319cc0-02e1-42dd-9487-a32e1a97caa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:34.691408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f319cc0-02e1-42dd-9487-a32e1a97caa5 None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=3b75c12a00304f15b7b175a19b8c096f, user.id=c5b5e18d6a8140dd843d4b21e38a9e8b) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:34.691811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f319cc0-02e1-42dd-9487-a32e1a97caa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:34.693383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f319cc0-02e1-42dd-9487-a32e1a97caa5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:34.701922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 606/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:54:34 2022] GET /identity/v3/role_assignments?user.id=c5b5e18d6a8140dd843d4b21e38a9e8b&scope.project.id=3b75c12a00304f15b7b175a19b8c096f&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:54:37.019784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373edc-2466-4660-9e78-554dff102600 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:37.020379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373edc-2466-4660-9e78-554dff102600 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:37.020379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373edc-2466-4660-9e78-554dff102600 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:37.021696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 580/1186] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:37.028299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e4f4-7265-4fde-9e23-a6e44cd28e14 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:37.028847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e4f4-7265-4fde-9e23-a6e44cd28e14 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:37.029250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e4f4-7265-4fde-9e23-a6e44cd28e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:37.052682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4ec1e4f4-7265-4fde-9e23-a6e44cd28e14 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:37.054769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4ec1e4f4-7265-4fde-9e23-a6e44cd28e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:37.143014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 607/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:37.428013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373edc-2466-4660-9e78-554dff102600 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:37.480105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:37.592416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373edc-2466-4660-9e78-554dff102600 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:37.655520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:37.656184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:37.656595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:37.657047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:37.674510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:54:37.681123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:54:37.682831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:54:37.684628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 581/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:54:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:54:37.689623 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1e4f4-7265-4fde-9e23-a6e44cd28e14 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:37.785885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:37.824642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1e4f4-7265-4fde-9e23-a6e44cd28e14 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:37.875160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:37.876216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:37.876711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:37.877190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:37.878762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:37.879321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:37.882038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:37.899003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 608/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:54:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:37.905500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:38.032044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:38.043448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b72ef-3afe-48e8-89c9-6c675915d137 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:38.091807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:38.092492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:38.092770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:38.093027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] PATH_INFO: `/v3/users/c5b5e18d6a8140dd843d4b21e38a9e8b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:38.094221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] RBAC: Authorizing `identity:get_user(user_id=c5b5e18d6a8140dd843d4b21e38a9e8b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:38.095797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:38.097505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 582/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:54:37 2022] GET /identity/v3/users/c5b5e18d6a8140dd843d4b21e38a9e8b => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:38.104200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:38.258314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343931b-0dbd-4463-aff9-8cad9c41a750 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:38.310851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:38.317702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be558cb9-d9eb-405f-87ff-d82f0718693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:38.318387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be558cb9-d9eb-405f-87ff-d82f0718693d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:38.318678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be558cb9-d9eb-405f-87ff-d82f0718693d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:38.319054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be558cb9-d9eb-405f-87ff-d82f0718693d None admin] PATH_INFO: `/v3/projects/3b75c12a00304f15b7b175a19b8c096f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:38.320439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be558cb9-d9eb-405f-87ff-d82f0718693d None admin] RBAC: Authorizing `identity:get_project(project_id=3b75c12a00304f15b7b175a19b8c096f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:38.321992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be558cb9-d9eb-405f-87ff-d82f0718693d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:38.323818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 609/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:54:38 2022] GET /identity/v3/projects/3b75c12a00304f15b7b175a19b8c096f => generated 361 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:38.330518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:38.499631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b2d21-243d-4ee5-8cd2-1a8b85b8a4e8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:38.554202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a76781-3d96-4f58-9120-ac4b98b5b681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:38.555096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76781-3d96-4f58-9120-ac4b98b5b681 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:38.555375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76781-3d96-4f58-9120-ac4b98b5b681 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:38.555665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a76781-3d96-4f58-9120-ac4b98b5b681 None admin] PATH_INFO: `/v3/projects/3b75c12a00304f15b7b175a19b8c096f/users/c5b5e18d6a8140dd843d4b21e38a9e8b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:38.557800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a76781-3d96-4f58-9120-ac4b98b5b681 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b75c12a00304f15b7b175a19b8c096f, user_id=c5b5e18d6a8140dd843d4b21e38a9e8b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:38.559308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a76781-3d96-4f58-9120-ac4b98b5b681 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:38.580099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 583/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:54:38 2022] PUT /identity/v3/projects/3b75c12a00304f15b7b175a19b8c096f/users/c5b5e18d6a8140dd843d4b21e38a9e8b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:54:40.872078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fafa12-c373-4231-a887-aa98a74ac8df None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:40.872660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fafa12-c373-4231-a887-aa98a74ac8df None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:40.872987 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fafa12-c373-4231-a887-aa98a74ac8df None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:40.875278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 610/1193] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:54:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:54:40.882883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851f59c-e25b-4ffd-b7f3-2b7171d61549 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:40.883430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851f59c-e25b-4ffd-b7f3-2b7171d61549 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:40.883950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851f59c-e25b-4ffd-b7f3-2b7171d61549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:40.913094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d851f59c-e25b-4ffd-b7f3-2b7171d61549 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:54:40.915292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d851f59c-e25b-4ffd-b7f3-2b7171d61549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:54:40.941758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:41.025708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 584/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:54:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:54:41.512999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fafa12-c373-4231-a887-aa98a74ac8df None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:41.715119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fafa12-c373-4231-a887-aa98a74ac8df None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:41.762848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:41.763550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:41.763875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:41.764193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:41.781145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:54:41.786650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:54:41.788430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:54:41.790176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 611/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:54:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:54:41.798616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851f59c-e25b-4ffd-b7f3-2b7171d61549 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:41.943108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851f59c-e25b-4ffd-b7f3-2b7171d61549 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:41.996929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:41.997948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:41.998454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:41.998932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:42.000186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:42.000735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:42.003714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:42.019827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 585/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:54:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:42.027694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:42.115149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:42.191702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965ffd-b406-4b95-a862-88b8f08f3a31 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:42.248348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:42.248965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:42.249246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:42.249519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] PATH_INFO: `/v3/users/c5b5e18d6a8140dd843d4b21e38a9e8b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:42.250871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] RBAC: Authorizing `identity:get_user(user_id=c5b5e18d6a8140dd843d4b21e38a9e8b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:42.253437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:42.256129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 612/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 12:54:42 2022] GET /identity/v3/users/c5b5e18d6a8140dd843d4b21e38a9e8b => generated 311 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:42.262987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:42.280859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:42.389157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88aa2-766c-4b9a-9670-01620b3f3b7d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:42.447274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:42.447891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:42.448170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:42.448439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:42.536269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:42.537838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:42.553737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:42.654781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 586/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:54:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:54:42.662563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:42.832052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1ca67-593b-470a-bb60-4794266658f3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:42.881347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:42.891930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fb7b8-1d91-41ca-8149-e438d89e6666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:42.892585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb7b8-1d91-41ca-8149-e438d89e6666 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:42.892858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb7b8-1d91-41ca-8149-e438d89e6666 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:42.893130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fb7b8-1d91-41ca-8149-e438d89e6666 None admin] PATH_INFO: `/v3/projects/3b75c12a00304f15b7b175a19b8c096f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:42.894428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fb7b8-1d91-41ca-8149-e438d89e6666 None admin] RBAC: Authorizing `identity:get_project(project_id=3b75c12a00304f15b7b175a19b8c096f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:42.895933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fb7b8-1d91-41ca-8149-e438d89e6666 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:42.898059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 613/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 12:54:42 2022] GET /identity/v3/projects/3b75c12a00304f15b7b175a19b8c096f => generated 361 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:54:42.904558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:54:43.033285 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aa87c-2d1c-4199-9327-a520a0d491d7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:54:43.048903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:43.073808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9369-a913-4b96-b8d8-7c0011fdb48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:54:43.074638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9369-a913-4b96-b8d8-7c0011fdb48a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:54:43.075059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9369-a913-4b96-b8d8-7c0011fdb48a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:54:43.075418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9369-a913-4b96-b8d8-7c0011fdb48a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:54:43.076352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b9369-a913-4b96-b8d8-7c0011fdb48a None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=3b75c12a00304f15b7b175a19b8c096f, user.id=c5b5e18d6a8140dd843d4b21e38a9e8b) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:54:43.076745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b9369-a913-4b96-b8d8-7c0011fdb48a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:54:43.078430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b9369-a913-4b96-b8d8-7c0011fdb48a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:54:43.085728 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 587/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:54:42 2022] GET /identity/v3/role_assignments?user.id=c5b5e18d6a8140dd843d4b21e38a9e8b&scope.project.id=3b75c12a00304f15b7b175a19b8c096f&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:54:52.887074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:54:53.082189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=12 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:48.900069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cef32-6f95-4e1b-8277-a283c3e59757 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:48.900705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cef32-6f95-4e1b-8277-a283c3e59757 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:48.900808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cef32-6f95-4e1b-8277-a283c3e59757 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:48.902524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 614/1201] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:55:48.908050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc0a71-6813-4674-a9b9-50749afaf6ba None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:48.908636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc0a71-6813-4674-a9b9-50749afaf6ba None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:48.908636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc0a71-6813-4674-a9b9-50749afaf6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:48.930252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b5fc0a71-6813-4674-a9b9-50749afaf6ba None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:55:48.932403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b5fc0a71-6813-4674-a9b9-50749afaf6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:55:49.041722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 588/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:55:49.362198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cef32-6f95-4e1b-8277-a283c3e59757 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:49.541037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cef32-6f95-4e1b-8277-a283c3e59757 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:49.593885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:49.594671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:49.594933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:49.595233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:49.611782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:49.613531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:55:49.632151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:49.633510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:55:49.635474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:55:49.635474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:49.635474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:55:49.635474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 615/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:55:49.641099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc0a71-6813-4674-a9b9-50749afaf6ba None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:49.787512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc0a71-6813-4674-a9b9-50749afaf6ba None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:49.834200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:49.841909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c9ff-bc48-4d7a-8190-736f801f207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:49.842823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c9ff-bc48-4d7a-8190-736f801f207d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:49.843390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c9ff-bc48-4d7a-8190-736f801f207d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:49.843846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c9ff-bc48-4d7a-8190-736f801f207d None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:49.844761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c9ff-bc48-4d7a-8190-736f801f207d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:49.845134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c9ff-bc48-4d7a-8190-736f801f207d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:49.846760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c9ff-bc48-4d7a-8190-736f801f207d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:49.867700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 589/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:55:49.875425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:49.976151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:50.056481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2fe302-82be-468e-a197-9e7cbfa475e8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:50.100859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cec81-d0dd-4868-a3fa-97ab36fd7e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:50.101900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cec81-d0dd-4868-a3fa-97ab36fd7e4a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:50.102397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cec81-d0dd-4868-a3fa-97ab36fd7e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:50.102981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cec81-d0dd-4868-a3fa-97ab36fd7e4a None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:50.103916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cec81-d0dd-4868-a3fa-97ab36fd7e4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:50.105631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cec81-d0dd-4868-a3fa-97ab36fd7e4a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:50.155059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 616/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 12:55:49 2022] POST /identity/v3/users => generated 251 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:55:52.950875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a7d48-40e6-4bdf-9f3d-5850e07e10bf None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:52.951453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a7d48-40e6-4bdf-9f3d-5850e07e10bf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:52.951453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a7d48-40e6-4bdf-9f3d-5850e07e10bf None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:52.952775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 590/1206] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:55:52.963461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeefc8a-78be-4d98-8390-ddf15633739c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:52.964029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeefc8a-78be-4d98-8390-ddf15633739c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:52.964451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeefc8a-78be-4d98-8390-ddf15633739c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:52.990658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bdeefc8a-78be-4d98-8390-ddf15633739c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:55:52.991960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bdeefc8a-78be-4d98-8390-ddf15633739c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:55:53.099904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 617/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:55:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:55:53.411282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a7d48-40e6-4bdf-9f3d-5850e07e10bf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:53.564604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a7d48-40e6-4bdf-9f3d-5850e07e10bf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:53.633567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:53.633822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:53.634943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:53.635562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:53.655229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:55:53.665133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:53.666813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:55:53.668139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 591/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:55:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:55:53.674997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeefc8a-78be-4d98-8390-ddf15633739c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:53.747021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:53.822596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeefc8a-78be-4d98-8390-ddf15633739c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:53.869359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:53.869359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:53.872169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:53.872169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:53.872169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:53.872169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:53.872500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:53.895840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 618/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:55:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:55:53.902679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:53.984129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:54.058633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51d97e-0492-4c9d-b541-587df0ef8bcf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:54.108520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:54.109558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:54.110151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:54.110514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:54.125764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:54.127311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:55:54.144317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:54.145588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:55:54.146604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:55:54.146604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:54.146604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:55:54.146604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 592/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:55:54.151957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:54.289600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:54.293320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e37398-af6a-416b-a160-c0fafe0f27b4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:54.340234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:54.341053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:54.341400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:54.341723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:54.342577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:54.342821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:54.344500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:54.362481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 619/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:55:54.370334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:54.527560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4380b3-dfc3-4b3f-929a-76886deecaeb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:54.529978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:54.571723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:54.572791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:54.573241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:54.573630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:54.582941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:54.584394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:55:54.597756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:54.598955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:55:54.600164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:55:54.602511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 593/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:55:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:55:54.608673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:54.751004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8fc2b-e8f5-4fe9-acb2-742698572207 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:54.791232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:54.791841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:54.792119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:54.792414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:54.793167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:54.793456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:54.795013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:54.818337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 620/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 12:55:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:55:54.824616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:54.947710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975d925-bafc-425e-96d0-3a624b507c95 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:54.996191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:54.997927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:54.998288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:54.998553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:55.037485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:55.084324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:55.085885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:55.216814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 594/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:55:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:55:55.222701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:55.245831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:55.347461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d64b4-eeb9-4063-806f-7ce8e76431dd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:55.403191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:55.404557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:55.406391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:55.406391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:55.421855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:55.424013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:55:55.446332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:55.448541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:55:55.449890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:55:55.449890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:55.449890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:55:55.449890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 621/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:55:55.454623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:55.550533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:55.620713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cc494-bb98-445b-944a-4008ffe8fb27 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:55.665938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:55.666986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:55.667257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:55.667537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:55.668312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:55.668619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:55.670263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:55.689339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 595/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:55:55.695931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:55.788292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:55.831507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e86ac-777d-49a9-8131-ef739060a2c4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:55.874237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:55.875039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:55.875454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:55.875877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:55.888669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:55.891281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:55:55.908338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:55.910462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:55:55.912363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 622/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:55:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:55:55.916489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:56.068281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c01660-1e3d-45b1-9b49-b5dc13ddf60e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:56.093743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:56.107443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03747f21-59d3-4d33-b635-3f1f261a8514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:56.108106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03747f21-59d3-4d33-b635-3f1f261a8514 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:56.108383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03747f21-59d3-4d33-b635-3f1f261a8514 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:56.108651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03747f21-59d3-4d33-b635-3f1f261a8514 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:56.109350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03747f21-59d3-4d33-b635-3f1f261a8514 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:56.109653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03747f21-59d3-4d33-b635-3f1f261a8514 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:56.111174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03747f21-59d3-4d33-b635-3f1f261a8514 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:56.125148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 596/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:55:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:55:56.131629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:56.261569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aeb1e-cac2-48eb-ab8e-60d12c245639 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:56.296001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:56.302374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d72e97-efd4-45ee-9e2c-9080f80a0d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:56.303049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72e97-efd4-45ee-9e2c-9080f80a0d2e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:56.303329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72e97-efd4-45ee-9e2c-9080f80a0d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:56.303645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72e97-efd4-45ee-9e2c-9080f80a0d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:56.304495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d72e97-efd4-45ee-9e2c-9080f80a0d2e None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1d0c0be08934466c858ee214b06538b2) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:56.304805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d72e97-efd4-45ee-9e2c-9080f80a0d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:56.306746 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d72e97-efd4-45ee-9e2c-9080f80a0d2e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:56.313105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 623/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:55:56 2022] GET /identity/v3/role_assignments?user.id=1d0c0be08934466c858ee214b06538b2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:55:58.623779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2825a2-829b-42f7-81eb-a643f526df8d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:58.624396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2825a2-829b-42f7-81eb-a643f526df8d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:58.624396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2825a2-829b-42f7-81eb-a643f526df8d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:58.625977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 597/1220] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:55:58.633141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8094b-460f-47b4-a0bc-89c6b3ac5598 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:58.633649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8094b-460f-47b4-a0bc-89c6b3ac5598 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:58.633649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8094b-460f-47b4-a0bc-89c6b3ac5598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:58.655279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e6e8094b-460f-47b4-a0bc-89c6b3ac5598 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:55:58.656640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e6e8094b-460f-47b4-a0bc-89c6b3ac5598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:55:58.768026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 624/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:55:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:55:59.045969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2825a2-829b-42f7-81eb-a643f526df8d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:59.227989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2825a2-829b-42f7-81eb-a643f526df8d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:59.268821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:59.269457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:59.269806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:59.270082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:59.281481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:55:59.286370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:59.287411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:55:59.288583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 598/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:55:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:55:59.294324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8094b-460f-47b4-a0bc-89c6b3ac5598 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:59.423180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8094b-460f-47b4-a0bc-89c6b3ac5598 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:59.456312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:59.470857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:59.471542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:59.471871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:59.472170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:59.473028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:59.473339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:59.474799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:59.485201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 625/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:55:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:55:59.491873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:59.546514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:55:59.644022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1787dd-9f7c-4192-bec9-97b9e1e6ab93 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:59.687631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:59.688296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:59.688575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:59.688889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:59.701085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:59.702732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:55:59.722665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:55:59.725074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:55:59.726697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:55:59.726697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:55:59.726697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:55:59.726697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 599/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:55:59.733826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:55:59.868327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b3fe4-8103-4e30-9497-b28dd8b3b895 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:55:59.905402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:55:59.905954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:55:59.906233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:55:59.906509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:55:59.907148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:55:59.907450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:55:59.908838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:55:59.923714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 626/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:55:59.933960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:00.063042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:00.070335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb1c5b-6444-4667-b87e-e64e9712cb1c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:00.126364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:00.128075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:00.128386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:00.128663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:00.138982 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:00.140422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:56:00.154994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:00.156307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:00.157507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 600/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:55:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:00.163248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:00.221590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:00.287211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aae2b-e2c0-4cc0-a630-cccb7d6c38ca None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:00.326852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:00.327453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:00.327762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:00.327986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:00.328716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:00.329013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:00.330468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:00.349587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 627/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 12:56:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:56:00.356466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:00.473256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a997-aaac-4d32-aa26-2c66b618278a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:00.513990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:00.514798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:00.515175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:00.515541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:00.530337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:00.532626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:00.555813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:00.558314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:00.560387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:00.560387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:00.560387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:00.560387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 601/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:00.565813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:00.720028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092bb59-2a08-4daa-9596-b5ea1883efa7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:00.723781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:00.760498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:00.761207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:00.761415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:00.761835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:00.762434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:00.762747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:00.764151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:00.776851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 628/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:00.783072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:00.793567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:00.934398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecc491-d314-49bb-b789-ef44651430e0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:00.974597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:00.975257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:00.975639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:00.975897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:00.984608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:00.986117 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:56:00.999332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:01.003697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:01.005161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 602/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:01.006913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:01.139216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf890c-a789-49ad-bc58-cfbadc024e3f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:01.178813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5d79e-915b-4c4d-9ed7-01324e74242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:01.179479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5d79e-915b-4c4d-9ed7-01324e74242d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:01.179796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5d79e-915b-4c4d-9ed7-01324e74242d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:01.180143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5d79e-915b-4c4d-9ed7-01324e74242d None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:01.181032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5d79e-915b-4c4d-9ed7-01324e74242d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:01.181738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5d79e-915b-4c4d-9ed7-01324e74242d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:01.184379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5d79e-915b-4c4d-9ed7-01324e74242d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:01.199751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 629/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:56:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:01.206381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:01.285912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:01.338810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97392-f982-4a9f-a7e6-e0db482220fd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:01.383573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb5580-d55a-4203-86ef-e3de6b4e236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:01.384288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb5580-d55a-4203-86ef-e3de6b4e236a None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:01.384783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb5580-d55a-4203-86ef-e3de6b4e236a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:01.385063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb5580-d55a-4203-86ef-e3de6b4e236a None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1d0c0be08934466c858ee214b06538b2/roles/bca79ab169d64049a6983bc348d0c161` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:01.407837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb5580-d55a-4203-86ef-e3de6b4e236a None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=1d0c0be08934466c858ee214b06538b2, role_id=bca79ab169d64049a6983bc348d0c161)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:01.409296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb5580-d55a-4203-86ef-e3de6b4e236a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:01.430954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 603/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:56:01 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1d0c0be08934466c858ee214b06538b2/roles/bca79ab169d64049a6983bc348d0c161 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:56:03.772418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3523-13f1-43ff-b2ec-3fc29328b448 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:03.773045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3523-13f1-43ff-b2ec-3fc29328b448 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:03.773045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3523-13f1-43ff-b2ec-3fc29328b448 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:03.774435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 630/1233] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:03.781345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b455b5-06db-481a-b747-077ec3ead0ef None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:03.781918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b455b5-06db-481a-b747-077ec3ead0ef None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:03.781918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b455b5-06db-481a-b747-077ec3ead0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:03.801962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-24b455b5-06db-481a-b747-077ec3ead0ef None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:03.803200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-24b455b5-06db-481a-b747-077ec3ead0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:03.882321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 604/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:56:04.174885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3523-13f1-43ff-b2ec-3fc29328b448 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:04.203592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:04.290780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3523-13f1-43ff-b2ec-3fc29328b448 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:04.331585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:04.332350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:04.332641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:04.332913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:04.345394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:56:04.350721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:04.352130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:04.353200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 631/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 12:56:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:04.358967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b455b5-06db-481a-b747-077ec3ead0ef None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:04.431447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:04.512528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b455b5-06db-481a-b747-077ec3ead0ef None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:04.551816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:04.552604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:04.552990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:04.553363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:04.554162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:04.554566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:04.556041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:04.565854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 605/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 12:56:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:04.572419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:04.677566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:04.694882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525ab65-9eab-4f00-b87c-893676a8cfcb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:04.741042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:04.741752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:04.742011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:04.742282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:04.754736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:04.756237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:04.774678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:04.776511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:04.777452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:04.777452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:04.777452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:04.777452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 632/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:56:04.782911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:04.912494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c94bc-75ef-4934-afd6-5c85981fea05 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:04.923500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:04.952773 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:04.953327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:04.953607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:04.953911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:04.954582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:04.954932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:04.956563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:04.972677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 606/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:04.979013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:05.098325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08fb04-010d-4ea8-a615-4f72da699ad4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:05.136754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:05.137486 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:05.137851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:05.138290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:05.142811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:05.160017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:05.160017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:56:05.169391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:05.170703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:05.171870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:05.176588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 633/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:05.177842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:05.304315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f3c17-25f0-4907-a200-63eddb840f70 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:05.349187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:05.350084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:05.350441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:05.350731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:05.351461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:05.351775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:05.353485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:05.388064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 607/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 12:56:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:05.395646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:05.523115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fefbef-c64c-4054-853a-b8f737e65cb9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:05.560846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:05.561440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:05.561755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:05.562052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:05.629435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:05.643405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:05.645768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:05.780318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 634/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:56:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:56:05.786910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:05.822044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:05.914331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a58263-27e5-4db7-90c4-f03c12c8940f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:05.966887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:05.967507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:05.967793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:05.968066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:05.981289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:05.982975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:06.005746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:06.008214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:06.011322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:06.011322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:06.011322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:06.011322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 608/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:06.015745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:06.137787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:06.146356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bba21c-c904-4ab3-a386-20719876c469 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:06.196888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:06.197849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:06.198010 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:06.198247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:06.198874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:06.199174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:06.200602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:06.214314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 635/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:56:06.220680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:06.358801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:06.396563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372334-a6c2-4b58-b96f-54ac36d36797 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:06.436626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:06.437532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:06.437862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:06.438283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:06.446745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:06.448504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:56:06.460709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:06.462082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:06.465348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 609/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:06.468601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:06.590672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a2356-e1d1-4aa0-a237-092db9d58e30 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:06.629029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd03a2f-ec29-464d-bdcf-52dca4edcd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:06.629733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd03a2f-ec29-464d-bdcf-52dca4edcd30 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:06.630069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd03a2f-ec29-464d-bdcf-52dca4edcd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:06.630349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd03a2f-ec29-464d-bdcf-52dca4edcd30 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:06.631074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd03a2f-ec29-464d-bdcf-52dca4edcd30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:06.631381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd03a2f-ec29-464d-bdcf-52dca4edcd30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:06.633214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd03a2f-ec29-464d-bdcf-52dca4edcd30 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:06.648073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 636/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:56:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:06.654729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:06.771866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226a086-d965-4e96-a123-8c8e1108b04b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:06.811288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c05683-1cc0-4a00-9d53-646afd9dee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:06.812013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c05683-1cc0-4a00-9d53-646afd9dee22 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:06.812254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c05683-1cc0-4a00-9d53-646afd9dee22 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:06.812532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c05683-1cc0-4a00-9d53-646afd9dee22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:06.813661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c05683-1cc0-4a00-9d53-646afd9dee22 None admin] RBAC: Adding query filter params (role.id=bca79ab169d64049a6983bc348d0c161, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1d0c0be08934466c858ee214b06538b2) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:06.814007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c05683-1cc0-4a00-9d53-646afd9dee22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:06.815538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c05683-1cc0-4a00-9d53-646afd9dee22 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:06.825746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 610/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:56:06 2022] GET /identity/v3/role_assignments?user.id=1d0c0be08934466c858ee214b06538b2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=bca79ab169d64049a6983bc348d0c161 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:09.210897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8730e65-f952-44dc-a4a5-eb4d8bb30bda None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:09.211513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8730e65-f952-44dc-a4a5-eb4d8bb30bda None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:09.212064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8730e65-f952-44dc-a4a5-eb4d8bb30bda None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:09.215225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 637/1247] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:09.221751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb5ca6-93eb-443e-910f-362c8cf08dde None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:09.222281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb5ca6-93eb-443e-910f-362c8cf08dde None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:09.222398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb5ca6-93eb-443e-910f-362c8cf08dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:09.230290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:09.244980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dffb5ca6-93eb-443e-910f-362c8cf08dde None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:09.246329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dffb5ca6-93eb-443e-910f-362c8cf08dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:09.338338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 611/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:56:09.628250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8730e65-f952-44dc-a4a5-eb4d8bb30bda None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:09.651559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:09.763787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8730e65-f952-44dc-a4a5-eb4d8bb30bda None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:09.816471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:09.817226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:09.817549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:09.817941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:09.831992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:56:09.837990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:09.840374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:09.843028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 638/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:56:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:56:09.849788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb5ca6-93eb-443e-910f-362c8cf08dde None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:10.000653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb5ca6-93eb-443e-910f-362c8cf08dde None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:10.036945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:10.043220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:10.043811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:10.044109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:10.044364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:10.045037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:10.045340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:10.046807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:10.058978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 612/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:56:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:10.065636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:10.195712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:10.232460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2bb83-6806-407c-b021-fef85f6217f5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:10.287646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:10.288513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:10.288859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:10.289171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:10.302760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:10.304364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:10.322768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:10.327446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:10.328451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:10.328451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:10.328451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:10.328451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 639/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:10.330165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:10.460752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc61c2-956d-4de9-a4cb-82b9d4c2d40d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:10.502117 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:10.502783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:10.503224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:10.503632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:10.504369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:10.504680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:10.506364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:10.523240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 613/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:10.530213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:10.666604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384210b-ddbb-471b-9bee-3cdd5b0cac25 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:10.715358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:10.733615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:10.734287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:10.734607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:10.734845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:10.746294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:10.750702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:56:10.764774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:10.766620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:10.768117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 640/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:10.773961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:10.868728 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:10.951886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d9eb8-58b1-415a-9c3e-cfbc70864d87 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:10.999196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:10.999866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:11.000269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:11.000572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:11.001315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:11.001650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:11.003389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:11.026470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 614/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 12:56:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:11.033238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:11.186871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502f008-ba4f-4eb8-ab00-8dd01e6aa2a0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:11.236126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:11.236947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:11.237392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:11.237857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:11.322614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:11.341358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:11.344229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:11.493400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 641/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:56:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 12:56:11.501474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:11.628066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12d164-8709-45ff-bbec-500d876d0570 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:11.648304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:11.681732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:11.682765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:11.682765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:11.683062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:11.694288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:11.695799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:11.716138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:11.717539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:11.718863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:11.718863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:11.718863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:11.718863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 615/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:11.726638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:11.846788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:11.867298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e71fc-dbce-4b9f-8757-6861f38b7c7c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:11.918311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:11.919037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:11.919371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:11.919691 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:11.920481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:11.920838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:11.922728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:11.939898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 642/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:11.946110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:12.083118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc19a4-f8a6-4bbe-a535-083bfdbb3512 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:12.139953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:12.140656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:12.140941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:12.141455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:12.149323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:12.151013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:56:12.164645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:12.166762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:12.168823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 616/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:12.174164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:12.340846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf0ab4-ed74-4503-9c9e-8df3f6d96067 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:12.401408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cfa58-9932-41c0-a71a-eb6c9f998863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:12.402570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cfa58-9932-41c0-a71a-eb6c9f998863 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:12.403076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cfa58-9932-41c0-a71a-eb6c9f998863 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:12.403564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cfa58-9932-41c0-a71a-eb6c9f998863 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:12.404885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cfa58-9932-41c0-a71a-eb6c9f998863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:12.405539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cfa58-9932-41c0-a71a-eb6c9f998863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:12.408161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cfa58-9932-41c0-a71a-eb6c9f998863 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:12.434464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 643/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:12.443502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:12.491398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:12.599484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b52e2d-385e-4425-afc6-c67a13af4794 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:12.642802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006f532-d0e6-480b-890c-e7ee1c9e892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:12.643475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006f532-d0e6-480b-890c-e7ee1c9e892f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:12.643749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006f532-d0e6-480b-890c-e7ee1c9e892f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:12.644015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006f532-d0e6-480b-890c-e7ee1c9e892f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:12.644840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006f532-d0e6-480b-890c-e7ee1c9e892f None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1d0c0be08934466c858ee214b06538b2) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:12.645108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006f532-d0e6-480b-890c-e7ee1c9e892f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:12.646636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006f532-d0e6-480b-890c-e7ee1c9e892f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:12.654898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 617/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:56:12 2022] GET /identity/v3/role_assignments?user.id=1d0c0be08934466c858ee214b06538b2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:14.904832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997fcc-c378-429f-a0a2-2da1b36b7576 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:14.905391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997fcc-c378-429f-a0a2-2da1b36b7576 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:14.905391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997fcc-c378-429f-a0a2-2da1b36b7576 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:14.906761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 644/1261] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:14.915995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9865c-c3e5-4329-8946-3845aedd93ce None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:14.915995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9865c-c3e5-4329-8946-3845aedd93ce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:14.915995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9865c-c3e5-4329-8946-3845aedd93ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:14.933146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-05a9865c-c3e5-4329-8946-3845aedd93ce None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:14.934393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-05a9865c-c3e5-4329-8946-3845aedd93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:15.012061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 618/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:56:15.281983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24997fcc-c378-429f-a0a2-2da1b36b7576 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:15.293483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:15.404228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24997fcc-c378-429f-a0a2-2da1b36b7576 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:15.440736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:15.441359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:15.441632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:15.441931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:15.452192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:56:15.456946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:15.458018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:15.458920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 645/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:15.464377 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9865c-c3e5-4329-8946-3845aedd93ce None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:15.483436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:15.577320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9865c-c3e5-4329-8946-3845aedd93ce None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:15.613654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:15.614285 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:15.614567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:15.614835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:15.615509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:15.615810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:15.617179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:15.625890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 619/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:56:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:15.631711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:15.731410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:15.788931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54124cd-0501-4107-b4aa-c63c53ad7333 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:15.826636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:15.827752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:15.828136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:15.828613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:15.841661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:15.843227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:15.859706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:15.860977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:15.861990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:15.861990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:15.861990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:15.861990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 646/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:15.866927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:15.954224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:15.975460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed0cb1-c242-4ba2-b53f-ef8cd1769a1d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:16.013554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:16.014208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:16.014484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:16.014754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:16.015399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:16.015833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:16.017070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:16.029876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 620/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:16.036376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:16.144728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2f2ed-7871-4c2a-a4d3-0ebc36978b80 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:16.166572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:16.183198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:16.183865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:16.184141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:16.184410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:16.193315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:16.194828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:56:16.208098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:16.209216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:16.210841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 647/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:16.215683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:16.324251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48c203-87a4-4818-b6f5-2f88dd4805cf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:16.362529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:16.363580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:16.364121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:16.364576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:16.365937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:16.366499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:16.368592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:16.382917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:16.401789 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 621/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 12:56:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:16.407817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:16.521767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f092e-5119-4120-a34c-caa07614fbb7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:16.555245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:16.555876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:16.556157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:16.556421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:16.567436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:16.568887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:16.583825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:16.585081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:16.586242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:16.586242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:16.586242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:16.589904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 648/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:16.591118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:16.702578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e311269-188e-42ed-bab9-a6c6851cdd29 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:16.742407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:16.743080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:16.743807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:16.744084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:16.745617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:16.745951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:16.747361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:16.760991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 622/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:16.767678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:16.811803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:16.900197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074381-36d5-402f-8f5f-fc6466e69e28 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:16.941713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:16.942340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:16.942618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:16.942886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:16.950404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:16.951890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:56:16.964093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:16.965284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:16.966543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 649/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:16.973121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:17.048619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:17.101343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89a08d-d5a8-46d7-8689-8e6df7d6f6a4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:17.137648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eca0cd-77f8-47ff-a20e-93b5a5950c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:17.138287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eca0cd-77f8-47ff-a20e-93b5a5950c99 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:17.138564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eca0cd-77f8-47ff-a20e-93b5a5950c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:17.138863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eca0cd-77f8-47ff-a20e-93b5a5950c99 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:17.139547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eca0cd-77f8-47ff-a20e-93b5a5950c99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:17.139867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eca0cd-77f8-47ff-a20e-93b5a5950c99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:17.141639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eca0cd-77f8-47ff-a20e-93b5a5950c99 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:17.155554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 623/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:17.161747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:17.262787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:17.272408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497cf2a-77f1-4800-b67c-ca605b87965f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:17.315090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da4af-8e9a-4f7d-918a-87ee6e812c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:17.315756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da4af-8e9a-4f7d-918a-87ee6e812c88 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:17.316034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da4af-8e9a-4f7d-918a-87ee6e812c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:17.316304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da4af-8e9a-4f7d-918a-87ee6e812c88 None admin] PATH_INFO: `/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1d0c0be08934466c858ee214b06538b2/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:17.318272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da4af-8e9a-4f7d-918a-87ee6e812c88 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8f6b3545f548babea667c09cc5765a, user_id=1d0c0be08934466c858ee214b06538b2, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:17.319735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da4af-8e9a-4f7d-918a-87ee6e812c88 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:17.336515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 650/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 12:56:17 2022] PUT /identity/v3/projects/0d8f6b3545f548babea667c09cc5765a/users/1d0c0be08934466c858ee214b06538b2/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:56:19.555958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b42ac-acba-4d53-a3bd-3c7b0393dac8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:19.556630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b42ac-acba-4d53-a3bd-3c7b0393dac8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:19.556630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b42ac-acba-4d53-a3bd-3c7b0393dac8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:19.558674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 624/1274] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:19.564027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d4bb6-11d0-4bdd-94aa-d80e452e1ecc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:19.564489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d4bb6-11d0-4bdd-94aa-d80e452e1ecc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:19.564580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d4bb6-11d0-4bdd-94aa-d80e452e1ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:19.585996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-262d4bb6-11d0-4bdd-94aa-d80e452e1ecc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:19.587212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-262d4bb6-11d0-4bdd-94aa-d80e452e1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:19.661191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 651/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:56:19.924455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b42ac-acba-4d53-a3bd-3c7b0393dac8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:20.034202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b42ac-acba-4d53-a3bd-3c7b0393dac8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:20.052953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:20.071853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:20.072480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:20.072791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:20.073575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:20.084209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:56:20.089704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:20.093772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:20.095080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 625/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 12:56:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:20.097314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d4bb6-11d0-4bdd-94aa-d80e452e1ecc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:20.200453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:20.223470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d4bb6-11d0-4bdd-94aa-d80e452e1ecc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:20.266484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:20.267093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:20.267367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:20.267630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:20.268291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:20.268578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:20.269978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:20.279984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 652/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 12:56:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:20.286194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:20.402093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52341e14-8785-4104-af5d-1c00307c8f89 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:20.447393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:20.448209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:20.448531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:20.448923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:20.462192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:20.463780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:20.480169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:20.481345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:20.482529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:20.482529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:20.482529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:20.482529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 626/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:20.487285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:20.604224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a983-7b78-41b0-8626-44c25d0e5d7f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:20.641726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49760701-db12-48f6-964c-681653d83100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:20.642347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49760701-db12-48f6-964c-681653d83100 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:20.642625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49760701-db12-48f6-964c-681653d83100 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:20.642897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49760701-db12-48f6-964c-681653d83100 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:20.643546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49760701-db12-48f6-964c-681653d83100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:20.643843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49760701-db12-48f6-964c-681653d83100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:20.645226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49760701-db12-48f6-964c-681653d83100 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:20.658309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 653/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:20.664291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:20.697497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:20.776761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a444e-8294-4458-b2ec-77904d5382fe None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:20.814746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:20.815390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:20.815668 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:20.815974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:20.824821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:20.826635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 12:56:20.840902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:20.842405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:20.843702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 627/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:20.848895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49760701-db12-48f6-964c-681653d83100 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:20.853776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:20.969659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49760701-db12-48f6-964c-681653d83100 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:21.010136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:21.011008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:21.011294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:21.011546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:21.012303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:21.012613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:21.014091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:21.033938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 654/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 12:56:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:21.040038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:21.145428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:21.155937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3f2b7-49ba-4877-ac22-1cd09016a79f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:21.198016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:21.199077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:21.199077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:21.199297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:21.274152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:21.275618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:21.373925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:21.387017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 628/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 12:56:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 12:56:21.393306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:21.470182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:21.519822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93373b-9211-4879-8f2d-9b6753e1f4ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:21.562233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:21.562895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:21.563152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:21.563455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:21.576054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:21.577557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:56:21.593710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:21.594912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:56:21.595883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:56:21.595883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:56:21.595883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:21.595883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 655/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 12:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:21.601146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:21.716527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c14b07-f631-4edc-984d-ec705177b371 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:21.756456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:21.757177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:21.757505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:21.757856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:21.758584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:21.759261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:21.761701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:21.775892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 629/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 12:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:21.782383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:21.913424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40112f9-44e5-403e-a75c-7e37ff218ad2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:21.930058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:21.954615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:21.955359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:21.955642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:21.955916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:21.964430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:21.965950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 12:56:21.979036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:21.980299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Sep 07 12:56:21.981614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 656/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 12:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:21.986957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:22.041064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:22.095953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f1b5a-3908-4b1d-9431-0767d593d4af None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:22.129308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219eead-852d-4649-98bd-11cc0fef9945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:22.129944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219eead-852d-4649-98bd-11cc0fef9945 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:22.130224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219eead-852d-4649-98bd-11cc0fef9945 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:22.130493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219eead-852d-4649-98bd-11cc0fef9945 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:22.131174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219eead-852d-4649-98bd-11cc0fef9945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:22.131475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219eead-852d-4649-98bd-11cc0fef9945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:22.132905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219eead-852d-4649-98bd-11cc0fef9945 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:22.146140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 630/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 12:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:56:22.152634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:22.268285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350f5ce-dd6c-4fc3-9f49-c4fbe8c726f2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:22.303905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625217b-d216-45e7-b927-1012712e7bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:22.304513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625217b-d216-45e7-b927-1012712e7bfb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:22.304793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625217b-d216-45e7-b927-1012712e7bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:22.305065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625217b-d216-45e7-b927-1012712e7bfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:22.305818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625217b-d216-45e7-b927-1012712e7bfb None admin] RBAC: Adding query filter params (role.id=2cff20de7350442a92ef150718ecf5c0, scope.project.id=0d8f6b3545f548babea667c09cc5765a, user.id=1d0c0be08934466c858ee214b06538b2) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:22.306124 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625217b-d216-45e7-b927-1012712e7bfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:22.307616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625217b-d216-45e7-b927-1012712e7bfb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:22.313938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 657/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 12:56:22 2022] GET /identity/v3/role_assignments?user.id=1d0c0be08934466c858ee214b06538b2&scope.project.id=0d8f6b3545f548babea667c09cc5765a&role.id=2cff20de7350442a92ef150718ecf5c0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:24.523326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d88f5-05ad-4f33-ae6b-2d42c42c56e8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:24.523897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d88f5-05ad-4f33-ae6b-2d42c42c56e8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:24.523897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d88f5-05ad-4f33-ae6b-2d42c42c56e8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:24.525215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 631/1288] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:24.531344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf7796-db85-405a-bbee-3a67f1166a72 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:24.531897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf7796-db85-405a-bbee-3a67f1166a72 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:24.531897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf7796-db85-405a-bbee-3a67f1166a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:24.561913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c7bf7796-db85-405a-bbee-3a67f1166a72 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:24.563162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c7bf7796-db85-405a-bbee-3a67f1166a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:24.600686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:24.644018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 658/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:56:24.911103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d88f5-05ad-4f33-ae6b-2d42c42c56e8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:25.021890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d88f5-05ad-4f33-ae6b-2d42c42c56e8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:25.034511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:25.059983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:25.060629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:25.060959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:25.061171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:25.061813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:25.063192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:25.072343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:25.073524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:25.074753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 632/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:56:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:25.080736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf7796-db85-405a-bbee-3a67f1166a72 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:25.192298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf7796-db85-405a-bbee-3a67f1166a72 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:25.228781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7e585-fb4e-4c64-afcc-abe84d42e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:25.229538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7e585-fb4e-4c64-afcc-abe84d42e205 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:25.229848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7e585-fb4e-4c64-afcc-abe84d42e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:25.230119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7e585-fb4e-4c64-afcc-abe84d42e205 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:25.230913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7e585-fb4e-4c64-afcc-abe84d42e205 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:25.231217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7e585-fb4e-4c64-afcc-abe84d42e205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:25.232618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7e585-fb4e-4c64-afcc-abe84d42e205 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:25.238481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 659/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 12:56:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:25.244708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:25.359554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7b0bf-9732-44dc-b1e2-5904552fb718 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:25.399777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c22c4-8136-4509-9258-68aa4d31bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:25.400408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c22c4-8136-4509-9258-68aa4d31bb7f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:25.400697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c22c4-8136-4509-9258-68aa4d31bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:25.400986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c22c4-8136-4509-9258-68aa4d31bb7f None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:25.401853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c22c4-8136-4509-9258-68aa4d31bb7f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:25.402241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c22c4-8136-4509-9258-68aa4d31bb7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:25.403643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c22c4-8136-4509-9258-68aa4d31bb7f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:25.409358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 633/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 12:56:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:27.577703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279988e1-5ee4-46fe-84f6-321b55c4607a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:27.578280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279988e1-5ee4-46fe-84f6-321b55c4607a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:27.578280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279988e1-5ee4-46fe-84f6-321b55c4607a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:27.579567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 660/1293] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:27.585270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93208-d5a5-4a0f-a653-0fcd6b974adb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:27.585725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93208-d5a5-4a0f-a653-0fcd6b974adb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:27.585817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93208-d5a5-4a0f-a653-0fcd6b974adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:27.605635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-68a93208-d5a5-4a0f-a653-0fcd6b974adb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:27.606886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-68a93208-d5a5-4a0f-a653-0fcd6b974adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:27.640446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:27.679931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 634/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 12:56:27.946258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279988e1-5ee4-46fe-84f6-321b55c4607a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:27.978956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:28.055008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279988e1-5ee4-46fe-84f6-321b55c4607a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:28.092744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce0bfb-0f1f-4b80-bafa-adba7ee50681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:28.093356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce0bfb-0f1f-4b80-bafa-adba7ee50681 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:28.093630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce0bfb-0f1f-4b80-bafa-adba7ee50681 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:28.093934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce0bfb-0f1f-4b80-bafa-adba7ee50681 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:28.094568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce0bfb-0f1f-4b80-bafa-adba7ee50681 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:28.095955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce0bfb-0f1f-4b80-bafa-adba7ee50681 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:28.112999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 661/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 12:56:27 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:56:30.272876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9003cf-f10d-4af4-9f65-a4c564c05247 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:30.273761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9003cf-f10d-4af4-9f65-a4c564c05247 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:30.273761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9003cf-f10d-4af4-9f65-a4c564c05247 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:30.275423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 635/1296] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:30.281261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466afbe2-b0af-480f-816d-bd2502b7454d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:30.281771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466afbe2-b0af-480f-816d-bd2502b7454d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:30.281879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466afbe2-b0af-480f-816d-bd2502b7454d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:30.303942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-466afbe2-b0af-480f-816d-bd2502b7454d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:30.305140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-466afbe2-b0af-480f-816d-bd2502b7454d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:30.381380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 662/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:56:30.645545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9003cf-f10d-4af4-9f65-a4c564c05247 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:30.770899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9003cf-f10d-4af4-9f65-a4c564c05247 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:30.807525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:30.808730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:30.809015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:30.809289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:30.809899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:30.811270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:30.820218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:30.821462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:30.822659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 636/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:56:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:30.827845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466afbe2-b0af-480f-816d-bd2502b7454d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:30.848176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:30.942935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466afbe2-b0af-480f-816d-bd2502b7454d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:30.986149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:30.986770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:30.987051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:30.987320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:30.988223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:30.988543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:30.989967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:30.997167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 663/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 12:56:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:31.004120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:31.049925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:31.133817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfcccb-8fc9-4fbc-8578-e72fea68a25d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:31.171799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dfc98-0495-4bac-a22d-420326027079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:31.172425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dfc98-0495-4bac-a22d-420326027079 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:31.172853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dfc98-0495-4bac-a22d-420326027079 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:31.173202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dfc98-0495-4bac-a22d-420326027079 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:31.173932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dfc98-0495-4bac-a22d-420326027079 None admin] RBAC: Adding query filter params (interface=public, service_id=ffae29969f50410c9955743ae1f4c184, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:31.174242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dfc98-0495-4bac-a22d-420326027079 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:31.175606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dfc98-0495-4bac-a22d-420326027079 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:31.182387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 637/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 12:56:31 2022] GET /identity/v3/endpoints?service_id=ffae29969f50410c9955743ae1f4c184&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:31.188548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:31.289505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:31.313919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ddcb2-e909-492f-92ba-a717b04a66e5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:31.359779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a5bb-a1fb-4c09-9d22-e5c940409807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:31.360434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a5bb-a1fb-4c09-9d22-e5c940409807 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:31.360717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a5bb-a1fb-4c09-9d22-e5c940409807 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:31.360991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a5bb-a1fb-4c09-9d22-e5c940409807 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:31.361602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a5bb-a1fb-4c09-9d22-e5c940409807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:31.363028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a5bb-a1fb-4c09-9d22-e5c940409807 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:31.370770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 664/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 12:56:31 2022] GET /identity/v3/services => generated 2191 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 12:56:33.625202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d557b-7497-4b8b-8cb9-c8c4f97c74cb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:33.625959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d557b-7497-4b8b-8cb9-c8c4f97c74cb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:33.625959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d557b-7497-4b8b-8cb9-c8c4f97c74cb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:33.627358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 638/1302] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:56:33.633646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eefd00-0ae4-452a-8e57-62c675636b1c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:33.634116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eefd00-0ae4-452a-8e57-62c675636b1c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:33.634116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eefd00-0ae4-452a-8e57-62c675636b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:33.653598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d9eefd00-0ae4-452a-8e57-62c675636b1c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:56:33.654825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d9eefd00-0ae4-452a-8e57-62c675636b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:56:33.728398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 665/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:56:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 12:56:34.005831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d557b-7497-4b8b-8cb9-c8c4f97c74cb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:34.112420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:34.119546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d557b-7497-4b8b-8cb9-c8c4f97c74cb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:34.153578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:34.154235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:34.154511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:34.154779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:34.155359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:34.156717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:56:34.165521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:56:34.166774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 12:56:34.167906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Sep 07 12:56:34.169857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 639/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 12:56:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:56:34.172870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eefd00-0ae4-452a-8e57-62c675636b1c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:34.253815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:34.278198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eefd00-0ae4-452a-8e57-62c675636b1c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:34.312907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391bd1d-f99d-4f72-82c5-f4d14634d455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:34.313508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bd1d-f99d-4f72-82c5-f4d14634d455 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:34.313810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bd1d-f99d-4f72-82c5-f4d14634d455 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:34.314081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bd1d-f99d-4f72-82c5-f4d14634d455 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:34.314700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391bd1d-f99d-4f72-82c5-f4d14634d455 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:56:34.315000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391bd1d-f99d-4f72-82c5-f4d14634d455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:34.316377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391bd1d-f99d-4f72-82c5-f4d14634d455 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:34.321717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 666/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 12:56:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:56:34.327883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:56:34.433645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0a527-7a13-4a94-b025-27c4783e4cbe None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:56:34.469099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7d47a-5b40-4848-94e7-2ab88f3871e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:56:34.469714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7d47a-5b40-4848-94e7-2ab88f3871e6 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:56:34.470017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7d47a-5b40-4848-94e7-2ab88f3871e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:56:34.470289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7d47a-5b40-4848-94e7-2ab88f3871e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:56:34.470953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7d47a-5b40-4848-94e7-2ab88f3871e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:56:34.472319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7d47a-5b40-4848-94e7-2ab88f3871e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:56:34.493673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 640/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 12:56:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:56:44.318626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:56:44.489086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:43.549460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c8d8b-dd5e-407b-892c-c12702f42915 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:43.550386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c8d8b-dd5e-407b-892c-c12702f42915 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:43.550386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c8d8b-dd5e-407b-892c-c12702f42915 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:43.563023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc0d8a-370a-45ac-b4e4-54e00a8f8799 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:43.563913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc0d8a-370a-45ac-b4e4-54e00a8f8799 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:43.564071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc0d8a-370a-45ac-b4e4-54e00a8f8799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:43.570895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 667/1307] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:57:43 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:57:43.807038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f7fc0d8a-370a-45ac-b4e4-54e00a8f8799 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:57:43.811085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7fc0d8a-370a-45ac-b4e4-54e00a8f8799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:44.065817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 641/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 12:57:44.372255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be3c7b-5e9f-40bf-81f3-47b781cbfedd None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:44.372852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be3c7b-5e9f-40bf-81f3-47b781cbfedd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:44.373212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be3c7b-5e9f-40bf-81f3-47b781cbfedd None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:44.376176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 668/1309] 10.222.0.17 () {54 vars in 965 bytes} [Wed Sep 7 12:57:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:57:44.388482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb49b7-aaef-4f61-9cc1-2a1498928261 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:44.389429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb49b7-aaef-4f61-9cc1-2a1498928261 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:44.390775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb49b7-aaef-4f61-9cc1-2a1498928261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:44.526626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-67cb49b7-aaef-4f61-9cc1-2a1498928261 None None] MFA Rules not processed for user `4302574365514eb1b25bd8e62736a4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:57:44.528000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-67cb49b7-aaef-4f61-9cc1-2a1498928261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:44.559685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:44.657864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 642/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Wed Sep 7 12:57:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:57:44.665577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be3c7b-5e9f-40bf-81f3-47b781cbfedd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:57:44.667421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-48be3c7b-5e9f-40bf-81f3-47b781cbfedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:44.910901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be3c7b-5e9f-40bf-81f3-47b781cbfedd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:57:45.017441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db17daa-334f-4682-8a02-fe184e87ca94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:57:45.018382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db17daa-334f-4682-8a02-fe184e87ca94 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:45.018854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db17daa-334f-4682-8a02-fe184e87ca94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:45.019282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db17daa-334f-4682-8a02-fe184e87ca94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:45.021226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5db17daa-334f-4682-8a02-fe184e87ca94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:45.112645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:45.187007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db17daa-334f-4682-8a02-fe184e87ca94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:57:45.188510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db17daa-334f-4682-8a02-fe184e87ca94 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:57:45.211202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 669/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:57:48.057649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d5f16-968e-46ad-a0f2-f81cd195238a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:48.058251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d5f16-968e-46ad-a0f2-f81cd195238a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:48.058696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d5f16-968e-46ad-a0f2-f81cd195238a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:48.061345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 643/1312] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:57:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:57:48.070514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4863312-d8b3-4205-aa07-f2939835befb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:48.071546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4863312-d8b3-4205-aa07-f2939835befb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:48.071947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4863312-d8b3-4205-aa07-f2939835befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:48.168216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b4863312-d8b3-4205-aa07-f2939835befb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:57:48.169486 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b4863312-d8b3-4205-aa07-f2939835befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:48.294957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 670/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:57:48.583219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d5f16-968e-46ad-a0f2-f81cd195238a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:57:48.585145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e08d5f16-968e-46ad-a0f2-f81cd195238a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:48.708343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d5f16-968e-46ad-a0f2-f81cd195238a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:57:48.791127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:48.794214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa797b6-92cb-4b99-ac0a-22e4dd9821f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:57:48.795165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa797b6-92cb-4b99-ac0a-22e4dd9821f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:48.795672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa797b6-92cb-4b99-ac0a-22e4dd9821f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:48.795950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa797b6-92cb-4b99-ac0a-22e4dd9821f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:48.797862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-faa797b6-92cb-4b99-ac0a-22e4dd9821f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:48.928847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa797b6-92cb-4b99-ac0a-22e4dd9821f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:57:48.930298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa797b6-92cb-4b99-ac0a-22e4dd9821f8 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:57:48.950886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 644/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:57:51.560364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68dee2-a781-410d-aef0-81a1e59058d3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:51.561232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68dee2-a781-410d-aef0-81a1e59058d3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:51.561232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68dee2-a781-410d-aef0-81a1e59058d3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:51.563057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 671/1315] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:57:51.570869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4c08-8cc9-4d87-bc56-c4db85567725 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:51.571410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4c08-8cc9-4d87-bc56-c4db85567725 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:51.571410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4c08-8cc9-4d87-bc56-c4db85567725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:51.659141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-839b4c08-8cc9-4d87-bc56-c4db85567725 None None] MFA Rules not processed for user `08b02c5702264b2d9c30d73839dd83ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:57:51.660311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-839b4c08-8cc9-4d87-bc56-c4db85567725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:51.728354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:51.782599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 645/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:57:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:57:52.095605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68dee2-a781-410d-aef0-81a1e59058d3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:57:52.097990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae68dee2-a781-410d-aef0-81a1e59058d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:52.111869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:52.227040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68dee2-a781-410d-aef0-81a1e59058d3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:57:52.319292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f177d7-ddea-4100-87cf-e71a8ff96511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:57:52.319909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f177d7-ddea-4100-87cf-e71a8ff96511 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:52.320162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f177d7-ddea-4100-87cf-e71a8ff96511 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:52.320431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f177d7-ddea-4100-87cf-e71a8ff96511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:52.322224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43f177d7-ddea-4100-87cf-e71a8ff96511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:52.406296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:52.462159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f177d7-ddea-4100-87cf-e71a8ff96511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:57:52.463925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f177d7-ddea-4100-87cf-e71a8ff96511 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:57:52.486595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 672/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:57:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:57:55.521186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd333c-575a-49bc-9ee1-4816b688925e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:55.521823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd333c-575a-49bc-9ee1-4816b688925e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:55.521966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd333c-575a-49bc-9ee1-4816b688925e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:55.523520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 646/1318] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:57:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:57:55.529771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca8020-1645-4839-9eaf-5a27c6652850 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:55.530284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca8020-1645-4839-9eaf-5a27c6652850 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:55.530379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca8020-1645-4839-9eaf-5a27c6652850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:55.607076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9bca8020-1645-4839-9eaf-5a27c6652850 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:57:55.608318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9bca8020-1645-4839-9eaf-5a27c6652850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:55.727832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 673/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:57:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:57:56.036756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd333c-575a-49bc-9ee1-4816b688925e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:57:56.038727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5ccd333c-575a-49bc-9ee1-4816b688925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:56.185257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd333c-575a-49bc-9ee1-4816b688925e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:57:56.283037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b7d5b-e290-43ae-a276-02f590d32cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:57:56.284166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7d5b-e290-43ae-a276-02f590d32cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:56.284656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7d5b-e290-43ae-a276-02f590d32cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:56.285320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7d5b-e290-43ae-a276-02f590d32cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:56.288791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bc6b7d5b-e290-43ae-a276-02f590d32cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:56.301395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:56.466729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b7d5b-e290-43ae-a276-02f590d32cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:57:56.468234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b7d5b-e290-43ae-a276-02f590d32cc2 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:57:56.491940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 647/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:57:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:57:59.515271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c080d-8513-4570-8221-fc6138333014 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:59.515919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c080d-8513-4570-8221-fc6138333014 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:59.516026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c080d-8513-4570-8221-fc6138333014 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:59.518048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 674/1321] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:57:59.524745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb498-4dab-4dcd-ad14-6699a858298e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:57:59.525259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb498-4dab-4dcd-ad14-6699a858298e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:57:59.525412 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb498-4dab-4dcd-ad14-6699a858298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:57:59.608862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3c2bb498-4dab-4dcd-ad14-6699a858298e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:57:59.610262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3c2bb498-4dab-4dcd-ad14-6699a858298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:57:59.676702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:57:59.723916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 648/1322] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 12:58:00.023628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c080d-8513-4570-8221-fc6138333014 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:00.025861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-362c080d-8513-4570-8221-fc6138333014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:00.061956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:00.176494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c080d-8513-4570-8221-fc6138333014 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:00.277559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71f638-5232-462c-8547-c91e11bad7ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:00.277559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f638-5232-462c-8547-c91e11bad7ef service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:00.277904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f638-5232-462c-8547-c91e11bad7ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:00.278128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f638-5232-462c-8547-c91e11bad7ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:00.281042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1e71f638-5232-462c-8547-c91e11bad7ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:00.373915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:00.430291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71f638-5232-462c-8547-c91e11bad7ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:00.431979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71f638-5232-462c-8547-c91e11bad7ef service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:00.454333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 675/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:03.057811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40ad1-64b7-4c6f-96e4-61ea6ddcb23c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:03.058707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40ad1-64b7-4c6f-96e4-61ea6ddcb23c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:03.058707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40ad1-64b7-4c6f-96e4-61ea6ddcb23c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:03.060315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 649/1324] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:58:03.066695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028fb59-dac4-4f32-9be6-430a534906c9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:03.067225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028fb59-dac4-4f32-9be6-430a534906c9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:03.067333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028fb59-dac4-4f32-9be6-430a534906c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:03.144435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0028fb59-dac4-4f32-9be6-430a534906c9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:03.145564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0028fb59-dac4-4f32-9be6-430a534906c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:03.267627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 676/1325] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:03.577606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc40ad1-64b7-4c6f-96e4-61ea6ddcb23c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:03.579480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3bc40ad1-64b7-4c6f-96e4-61ea6ddcb23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:03.717506 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc40ad1-64b7-4c6f-96e4-61ea6ddcb23c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:03.823895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3c6b8-66c6-4358-8cfa-32ce2fec00d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:03.824440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c6b8-66c6-4358-8cfa-32ce2fec00d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:03.824716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c6b8-66c6-4358-8cfa-32ce2fec00d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:03.824986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c6b8-66c6-4358-8cfa-32ce2fec00d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:03.826724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ed3c6b8-66c6-4358-8cfa-32ce2fec00d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:03.848580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:03.961250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3c6b8-66c6-4358-8cfa-32ce2fec00d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:03.962767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3c6b8-66c6-4358-8cfa-32ce2fec00d0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:03.984222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 650/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 12:58:06.295791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3b24-62ce-443c-8447-66e01adfc0d4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:06.296373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3b24-62ce-443c-8447-66e01adfc0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:06.296373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3b24-62ce-443c-8447-66e01adfc0d4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:06.298708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 677/1327] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:06.306491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325701cc-5357-432a-9870-b1baff7b1f78 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:06.307013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325701cc-5357-432a-9870-b1baff7b1f78 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:06.307013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325701cc-5357-432a-9870-b1baff7b1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:06.382985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-325701cc-5357-432a-9870-b1baff7b1f78 None None] MFA Rules not processed for user `08b02c5702264b2d9c30d73839dd83ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:06.384222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-325701cc-5357-432a-9870-b1baff7b1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:06.453260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:06.493631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 651/1328] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:06.789652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3b24-62ce-443c-8447-66e01adfc0d4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:06.791228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-161f3b24-62ce-443c-8447-66e01adfc0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:06.831023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:06.934253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3b24-62ce-443c-8447-66e01adfc0d4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:07.027081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b676c4-fbf5-4f0b-9e66-9e7d759ec167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:07.027858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b676c4-fbf5-4f0b-9e66-9e7d759ec167 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:07.028132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b676c4-fbf5-4f0b-9e66-9e7d759ec167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:07.028404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b676c4-fbf5-4f0b-9e66-9e7d759ec167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:07.030163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-37b676c4-fbf5-4f0b-9e66-9e7d759ec167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:07.123063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:07.168665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b676c4-fbf5-4f0b-9e66-9e7d759ec167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:07.172423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b676c4-fbf5-4f0b-9e66-9e7d759ec167 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:07.197629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 678/1329] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:09.674955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16c87f-c83b-4ea7-9005-9fe822ec230e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:09.675510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16c87f-c83b-4ea7-9005-9fe822ec230e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:09.675510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16c87f-c83b-4ea7-9005-9fe822ec230e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:09.676737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 652/1330] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:09.682490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec948ffa-6046-49d1-b7b5-cc1acc2d733d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:09.682951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec948ffa-6046-49d1-b7b5-cc1acc2d733d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:09.683055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec948ffa-6046-49d1-b7b5-cc1acc2d733d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:09.753686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ec948ffa-6046-49d1-b7b5-cc1acc2d733d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:09.754945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ec948ffa-6046-49d1-b7b5-cc1acc2d733d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:09.858071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 679/1331] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:10.145419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec991ed7-32d5-40d8-afdb-19c8f505e07c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:10.145720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec991ed7-32d5-40d8-afdb-19c8f505e07c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:10.145991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec991ed7-32d5-40d8-afdb-19c8f505e07c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:10.147323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 653/1332] 10.222.0.17 () {54 vars in 965 bytes} [Wed Sep 7 12:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:10.153113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27488cd-3aa0-41f9-ac16-c8bea3f9f5e1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:10.153438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27488cd-3aa0-41f9-ac16-c8bea3f9f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:10.153792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27488cd-3aa0-41f9-ac16-c8bea3f9f5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:10.202186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:10.236757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b27488cd-3aa0-41f9-ac16-c8bea3f9f5e1 None None] MFA Rules not processed for user `4302574365514eb1b25bd8e62736a4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:10.237998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b27488cd-3aa0-41f9-ac16-c8bea3f9f5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:10.314824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 680/1333] 10.222.0.17 () {58 vars in 1062 bytes} [Wed Sep 7 12:58:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:10.324088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec991ed7-32d5-40d8-afdb-19c8f505e07c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:10.325634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ec991ed7-32d5-40d8-afdb-19c8f505e07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:10.442194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec991ed7-32d5-40d8-afdb-19c8f505e07c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:10.519286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69013cdb-6a75-4c82-9f12-a845eac14649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:10.519899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69013cdb-6a75-4c82-9f12-a845eac14649 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:10.520184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69013cdb-6a75-4c82-9f12-a845eac14649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:10.520449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69013cdb-6a75-4c82-9f12-a845eac14649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:10.522108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-69013cdb-6a75-4c82-9f12-a845eac14649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:10.550081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:10.638629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69013cdb-6a75-4c82-9f12-a845eac14649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:10.639969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69013cdb-6a75-4c82-9f12-a845eac14649 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:10.659389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 654/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:13.089434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7514e4b-6f97-4ca6-9244-7936fe9e47ba None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:13.090011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7514e4b-6f97-4ca6-9244-7936fe9e47ba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:13.090011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7514e4b-6f97-4ca6-9244-7936fe9e47ba None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:13.091953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 681/1335] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:13.098540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67c87d-cf97-4884-8d33-2550de85e5de None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:13.099235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67c87d-cf97-4884-8d33-2550de85e5de None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:13.099235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67c87d-cf97-4884-8d33-2550de85e5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:13.172406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-da67c87d-cf97-4884-8d33-2550de85e5de None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:13.173958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-da67c87d-cf97-4884-8d33-2550de85e5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:13.268573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:13.308588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 655/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:13.593599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7514e4b-6f97-4ca6-9244-7936fe9e47ba None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:13.595264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e7514e4b-6f97-4ca6-9244-7936fe9e47ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:13.715056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7514e4b-6f97-4ca6-9244-7936fe9e47ba None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:13.748182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:13.796561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d99a51-b043-4bc0-af30-10d61d608f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:13.797160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99a51-b043-4bc0-af30-10d61d608f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:13.797439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99a51-b043-4bc0-af30-10d61d608f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:13.797708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99a51-b043-4bc0-af30-10d61d608f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:13.799881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a8d99a51-b043-4bc0-af30-10d61d608f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:13.916026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d99a51-b043-4bc0-af30-10d61d608f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:13.917531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d99a51-b043-4bc0-af30-10d61d608f6d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:13.936723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 682/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:16.766839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f088eaf-edd0-42f1-9940-ee86b6d77433 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:16.767441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f088eaf-edd0-42f1-9940-ee86b6d77433 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:16.767441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f088eaf-edd0-42f1-9940-ee86b6d77433 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:16.769203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 656/1338] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:16.774807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a12b55-d16e-4516-95a7-7e7672e5a815 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:16.775284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a12b55-d16e-4516-95a7-7e7672e5a815 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:16.775376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a12b55-d16e-4516-95a7-7e7672e5a815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:16.844758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-29a12b55-d16e-4516-95a7-7e7672e5a815 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:16.846135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-29a12b55-d16e-4516-95a7-7e7672e5a815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:16.854846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:16.964546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 683/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:17.242707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f088eaf-edd0-42f1-9940-ee86b6d77433 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:17.244253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f088eaf-edd0-42f1-9940-ee86b6d77433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:17.352543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f088eaf-edd0-42f1-9940-ee86b6d77433 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:17.429707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dbd4e-a231-45ec-b71d-e7083eae6186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:17.430338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dbd4e-a231-45ec-b71d-e7083eae6186 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:17.430613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dbd4e-a231-45ec-b71d-e7083eae6186 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:17.430886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dbd4e-a231-45ec-b71d-e7083eae6186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:17.432544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c54dbd4e-a231-45ec-b71d-e7083eae6186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:17.472259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:17.547321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dbd4e-a231-45ec-b71d-e7083eae6186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:17.548914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dbd4e-a231-45ec-b71d-e7083eae6186 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:17.568647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 657/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:21.945217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec050cb-8a86-41a6-9a77-587cf532decd None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:21.945798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec050cb-8a86-41a6-9a77-587cf532decd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:21.945798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec050cb-8a86-41a6-9a77-587cf532decd None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:21.947128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 684/1341] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:21.952967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c0df-c73e-4253-8310-8dcf632e0490 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:21.953563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c0df-c73e-4253-8310-8dcf632e0490 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:21.953563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c0df-c73e-4253-8310-8dcf632e0490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:22.025325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3ef2c0df-c73e-4253-8310-8dcf632e0490 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:22.026507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3ef2c0df-c73e-4253-8310-8dcf632e0490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:22.125568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:22.139005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 658/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 12:58:22.430209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec050cb-8a86-41a6-9a77-587cf532decd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:22.431783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4ec050cb-8a86-41a6-9a77-587cf532decd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:22.555451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec050cb-8a86-41a6-9a77-587cf532decd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:22.609899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:22.654067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b4a7b-38a2-46a8-a844-36487b6909be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:22.654591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b4a7b-38a2-46a8-a844-36487b6909be service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:22.654865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b4a7b-38a2-46a8-a844-36487b6909be service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:22.655096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b4a7b-38a2-46a8-a844-36487b6909be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:22.656774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a4b4a7b-38a2-46a8-a844-36487b6909be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:22.789244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b4a7b-38a2-46a8-a844-36487b6909be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:22.790730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b4a7b-38a2-46a8-a844-36487b6909be service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:22.811378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 685/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:25.630954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b0e27-43e6-4aab-b54a-5e31e589302a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:25.631811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b0e27-43e6-4aab-b54a-5e31e589302a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:25.631811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b0e27-43e6-4aab-b54a-5e31e589302a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:25.633722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 659/1344] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:25.640214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c964d-6b24-4a41-951d-21779471fd19 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:25.640761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c964d-6b24-4a41-951d-21779471fd19 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:25.640761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c964d-6b24-4a41-951d-21779471fd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:25.723682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a59c964d-6b24-4a41-951d-21779471fd19 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:25.724937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a59c964d-6b24-4a41-951d-21779471fd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:25.747500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:25.843778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 686/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:26.131309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b0e27-43e6-4aab-b54a-5e31e589302a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:26.132998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-358b0e27-43e6-4aab-b54a-5e31e589302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:26.257254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b0e27-43e6-4aab-b54a-5e31e589302a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:26.348302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e362b-5881-4c0d-8c21-5913bbe79089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:26.348842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e362b-5881-4c0d-8c21-5913bbe79089 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:26.349123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e362b-5881-4c0d-8c21-5913bbe79089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:26.349397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e362b-5881-4c0d-8c21-5913bbe79089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:26.351114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d4e362b-5881-4c0d-8c21-5913bbe79089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:26.409808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:26.493152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e362b-5881-4c0d-8c21-5913bbe79089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:26.494647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e362b-5881-4c0d-8c21-5913bbe79089 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:26.522090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 660/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 12:58:31.561067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde896f-0d28-41e3-bbad-e8835da8fcfe None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:31.561795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde896f-0d28-41e3-bbad-e8835da8fcfe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:31.561795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde896f-0d28-41e3-bbad-e8835da8fcfe None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:31.563335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 687/1347] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:31.570706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731e84d-d72d-47cc-b6b1-1305a7de5389 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:31.571371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731e84d-d72d-47cc-b6b1-1305a7de5389 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:31.571371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731e84d-d72d-47cc-b6b1-1305a7de5389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:31.666497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9731e84d-d72d-47cc-b6b1-1305a7de5389 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:31.667759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9731e84d-d72d-47cc-b6b1-1305a7de5389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:31.840343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:31.845719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 661/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:32.203824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde896f-0d28-41e3-bbad-e8835da8fcfe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:32.205420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ffde896f-0d28-41e3-bbad-e8835da8fcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:32.334715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde896f-0d28-41e3-bbad-e8835da8fcfe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:32.398932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:32.439679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c753e70-e82d-4442-8237-c2a5d91230e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:32.440734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c753e70-e82d-4442-8237-c2a5d91230e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:32.441229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c753e70-e82d-4442-8237-c2a5d91230e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:32.441728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c753e70-e82d-4442-8237-c2a5d91230e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:32.444913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5c753e70-e82d-4442-8237-c2a5d91230e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:32.593904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c753e70-e82d-4442-8237-c2a5d91230e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:32.595289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c753e70-e82d-4442-8237-c2a5d91230e0 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:32.628136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 688/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:38.419549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ca0e-d130-46b8-86f1-304c0c37a0aa None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:38.420039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ca0e-d130-46b8-86f1-304c0c37a0aa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:38.420249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0ca0e-d130-46b8-86f1-304c0c37a0aa None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:38.421932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 662/1350] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 12:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:38.429073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b4370-ce7b-4a59-92c2-53098f300304 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:38.429742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b4370-ce7b-4a59-92c2-53098f300304 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:38.429742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b4370-ce7b-4a59-92c2-53098f300304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:38.564084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9b1b4370-ce7b-4a59-92c2-53098f300304 None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:38.566437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9b1b4370-ce7b-4a59-92c2-53098f300304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:38.605714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:38.777869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 689/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 12:58:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:43.892947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed814744-65f2-4fbe-9006-8c993a5a8e0a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:43.894119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed814744-65f2-4fbe-9006-8c993a5a8e0a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:43.894864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed814744-65f2-4fbe-9006-8c993a5a8e0a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:43.897468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 663/1352] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 12:58:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:43.902976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3043ff-63c6-4b19-8543-fe841bc56b63 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:43.903459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3043ff-63c6-4b19-8543-fe841bc56b63 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:43.903563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3043ff-63c6-4b19-8543-fe841bc56b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:43.999422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6c3043ff-63c6-4b19-8543-fe841bc56b63 None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:44.000637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6c3043ff-63c6-4b19-8543-fe841bc56b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:44.086328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:44.129585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 690/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:45.696500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cb7a1-5007-434f-a461-925e92016ce9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:45.697075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cb7a1-5007-434f-a461-925e92016ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:45.697075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cb7a1-5007-434f-a461-925e92016ce9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:45.699078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 664/1354] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 12:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:45.704424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c58ea-b8a1-45d1-8587-0d8836bc6719 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:45.704907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c58ea-b8a1-45d1-8587-0d8836bc6719 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:45.705012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c58ea-b8a1-45d1-8587-0d8836bc6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:45.790615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-078c58ea-b8a1-45d1-8587-0d8836bc6719 None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:45.791824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-078c58ea-b8a1-45d1-8587-0d8836bc6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:45.915638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 691/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 12:58:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:49.621742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b0e0-334e-40c0-883a-e5504b2a7b7b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:49.622295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b0e0-334e-40c0-883a-e5504b2a7b7b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:49.622295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b0e0-334e-40c0-883a-e5504b2a7b7b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:49.623522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 665/1356] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:49.630653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701b179-cf96-4673-921d-bc0f87ee5a56 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:49.631648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701b179-cf96-4673-921d-bc0f87ee5a56 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:49.632144 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701b179-cf96-4673-921d-bc0f87ee5a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:49.684514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:49.726460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8701b179-cf96-4673-921d-bc0f87ee5a56 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:49.727556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8701b179-cf96-4673-921d-bc0f87ee5a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:49.834455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 692/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:50.164920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53f81-a7fa-449b-8db1-478778120b6f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:50.165628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53f81-a7fa-449b-8db1-478778120b6f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:50.166162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53f81-a7fa-449b-8db1-478778120b6f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:50.168267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 666/1358] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 12:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:50.176416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b02b2-718a-4c81-a47e-8c4119ac8911 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:50.177016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b02b2-718a-4c81-a47e-8c4119ac8911 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:50.177790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b02b2-718a-4c81-a47e-8c4119ac8911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:50.263301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-839b02b2-718a-4c81-a47e-8c4119ac8911 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:50.264613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-839b02b2-718a-4c81-a47e-8c4119ac8911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:50.321757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:50.426093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 693/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 12:58:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:50.432680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53f81-a7fa-449b-8db1-478778120b6f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:50.434477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-24b53f81-a7fa-449b-8db1-478778120b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:50.603410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53f81-a7fa-449b-8db1-478778120b6f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:50.686523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:50.736952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69cecf-1f84-49ee-a9d5-f92238bfbd8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:50.737598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69cecf-1f84-49ee-a9d5-f92238bfbd8b service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:50.737998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69cecf-1f84-49ee-a9d5-f92238bfbd8b service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:50.741273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69cecf-1f84-49ee-a9d5-f92238bfbd8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:50.741273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7c69cecf-1f84-49ee-a9d5-f92238bfbd8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:50.866682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69cecf-1f84-49ee-a9d5-f92238bfbd8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:50.868368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69cecf-1f84-49ee-a9d5-f92238bfbd8b service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:50.890714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 667/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:53.392182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f972b9-a4f7-4f1a-a19a-533742d277be None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.392182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f972b9-a4f7-4f1a-a19a-533742d277be None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.392182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f972b9-a4f7-4f1a-a19a-533742d277be None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.392182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 694/1361] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:53.403594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb44d-6fc9-4663-8fbb-dbd4d9001608 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.404364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb44d-6fc9-4663-8fbb-dbd4d9001608 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.404364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdb44d-6fc9-4663-8fbb-dbd4d9001608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.518108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9cfdb44d-6fc9-4663-8fbb-dbd4d9001608 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:53.519669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9cfdb44d-6fc9-4663-8fbb-dbd4d9001608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:53.570994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:53.646137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 668/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:53.949831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f972b9-a4f7-4f1a-a19a-533742d277be None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:53.951469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b8f972b9-a4f7-4f1a-a19a-533742d277be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:54.192005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f972b9-a4f7-4f1a-a19a-533742d277be None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:54.207022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:54.333906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd9085-3df2-4859-8422-095e9ea95c13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:54.334532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd9085-3df2-4859-8422-095e9ea95c13 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:54.334852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd9085-3df2-4859-8422-095e9ea95c13 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:54.335124 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd9085-3df2-4859-8422-095e9ea95c13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.336913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a2dd9085-3df2-4859-8422-095e9ea95c13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:54.440711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd9085-3df2-4859-8422-095e9ea95c13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:54.443087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd9085-3df2-4859-8422-095e9ea95c13 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:54.463226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 695/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:57.255700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b932c4-ed4d-4ceb-b273-d9cec90617a5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.256235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b932c4-ed4d-4ceb-b273-d9cec90617a5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.256235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b932c4-ed4d-4ceb-b273-d9cec90617a5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.257810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 669/1364] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:58:57.263365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbeb8d-6620-400f-9e8b-af9aaf0a3933 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.263893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbeb8d-6620-400f-9e8b-af9aaf0a3933 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.263893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbeb8d-6620-400f-9e8b-af9aaf0a3933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.280574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:57.351214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-69fbeb8d-6620-400f-9e8b-af9aaf0a3933 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:57.351214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69fbeb8d-6620-400f-9e8b-af9aaf0a3933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:57.482669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 696/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:57.782418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fbe22-e498-4aa6-8f77-5f6384b6d956 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.782801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fbe22-e498-4aa6-8f77-5f6384b6d956 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.782999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fbe22-e498-4aa6-8f77-5f6384b6d956 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.784501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 670/1366] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 12:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:57.790546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac5c5-79b6-46c2-96b3-4d70cbcf6f39 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.790798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac5c5-79b6-46c2-96b3-4d70cbcf6f39 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.791247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac5c5-79b6-46c2-96b3-4d70cbcf6f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.886096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a10ac5c5-79b6-46c2-96b3-4d70cbcf6f39 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:57.887389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a10ac5c5-79b6-46c2-96b3-4d70cbcf6f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:57.928062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:58.028804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 697/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 12:58:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:58.039319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fbe22-e498-4aa6-8f77-5f6384b6d956 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:58.040856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-594fbe22-e498-4aa6-8f77-5f6384b6d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:58.214365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fbe22-e498-4aa6-8f77-5f6384b6d956 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:58.248032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:58.281170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8b361-e291-49fd-8ced-c5c01bcf89da None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.281758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8b361-e291-49fd-8ced-c5c01bcf89da None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.281758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8b361-e291-49fd-8ced-c5c01bcf89da None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.282988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 698/1368] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:58.288957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250cf64-6277-4c7c-9aaa-ca733b5f5dea None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.289263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250cf64-6277-4c7c-9aaa-ca733b5f5dea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.289514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250cf64-6277-4c7c-9aaa-ca733b5f5dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.374256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f250cf64-6277-4c7c-9aaa-ca733b5f5dea None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:58.375586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f250cf64-6277-4c7c-9aaa-ca733b5f5dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:58.376571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d45c80-54bf-47e8-87fb-a0b6d4768e58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:58.378086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d45c80-54bf-47e8-87fb-a0b6d4768e58 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.378562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d45c80-54bf-47e8-87fb-a0b6d4768e58 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.379527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d45c80-54bf-47e8-87fb-a0b6d4768e58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.381609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b9d45c80-54bf-47e8-87fb-a0b6d4768e58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:58.505570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:58.529716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 699/1369] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 12:58:58.530008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d45c80-54bf-47e8-87fb-a0b6d4768e58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:58.530008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d45c80-54bf-47e8-87fb-a0b6d4768e58 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:58.550712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 671/1370] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:58:58.858499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3e5bd-d419-4b4b-bc7d-4b1aa44c82a0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.858499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3e5bd-d419-4b4b-bc7d-4b1aa44c82a0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.858499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3e5bd-d419-4b4b-bc7d-4b1aa44c82a0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.869377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 700/1371] 10.222.0.17 () {54 vars in 964 bytes} [Wed Sep 7 12:58:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:58.874457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1bc3b-968a-4097-bde4-c927964fd5c6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.874741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1bc3b-968a-4097-bde4-c927964fd5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.875017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1bc3b-968a-4097-bde4-c927964fd5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.915687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:58.989118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e2b1bc3b-968a-4097-bde4-c927964fd5c6 None None] MFA Rules not processed for user `42f946fc56244e179973d6f846eb7548`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:58.990550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e2b1bc3b-968a-4097-bde4-c927964fd5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:59.088884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 672/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Sep 7 12:58:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:58:59.102995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3e5bd-d419-4b4b-bc7d-4b1aa44c82a0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:59.104948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-25b3e5bd-d419-4b4b-bc7d-4b1aa44c82a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:59.225624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3e5bd-d419-4b4b-bc7d-4b1aa44c82a0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:59.318811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035f86d-1e03-4e01-8fee-8780197a41ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:59.319678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035f86d-1e03-4e01-8fee-8780197a41ab service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:59.319678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035f86d-1e03-4e01-8fee-8780197a41ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:59.319979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035f86d-1e03-4e01-8fee-8780197a41ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:59.321873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b035f86d-1e03-4e01-8fee-8780197a41ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:59.385356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:59.474321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035f86d-1e03-4e01-8fee-8780197a41ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:59.475630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035f86d-1e03-4e01-8fee-8780197a41ab service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:59.502625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 701/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 12:59:00.856528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6edcc0-929b-4d1a-a2b2-f98087271ac8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:00.857105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6edcc0-929b-4d1a-a2b2-f98087271ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:00.857105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6edcc0-929b-4d1a-a2b2-f98087271ac8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:00.859077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 673/1374] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:00.867162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31156ad4-c2a6-4e15-ad7a-57851b63af3c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:00.867764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31156ad4-c2a6-4e15-ad7a-57851b63af3c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:00.867764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31156ad4-c2a6-4e15-ad7a-57851b63af3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:00.959351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-31156ad4-c2a6-4e15-ad7a-57851b63af3c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:00.960642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31156ad4-c2a6-4e15-ad7a-57851b63af3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:01.087764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 702/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 12:59:01.376743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6edcc0-929b-4d1a-a2b2-f98087271ac8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:01.379361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7d6edcc0-929b-4d1a-a2b2-f98087271ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:01.550379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:01.600002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6edcc0-929b-4d1a-a2b2-f98087271ac8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:01.740303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff3592-8bec-4cb5-b951-3c91870c5e32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:01.740896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3592-8bec-4cb5-b951-3c91870c5e32 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:01.741177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3592-8bec-4cb5-b951-3c91870c5e32 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:01.741450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3592-8bec-4cb5-b951-3c91870c5e32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:01.743600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ccff3592-8bec-4cb5-b951-3c91870c5e32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:01.829366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:01.843357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff3592-8bec-4cb5-b951-3c91870c5e32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:01.844993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff3592-8bec-4cb5-b951-3c91870c5e32 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:01.865283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 674/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:04.104665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f01f7-e412-41a1-97f0-32607850602f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:04.105260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f01f7-e412-41a1-97f0-32607850602f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:04.105260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f01f7-e412-41a1-97f0-32607850602f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:04.106813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 703/1377] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:04.112214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946d1b3-6d2a-4531-b80e-cf2ffaf2f95b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:04.113037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946d1b3-6d2a-4531-b80e-cf2ffaf2f95b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:04.113280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946d1b3-6d2a-4531-b80e-cf2ffaf2f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:04.182219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9946d1b3-6d2a-4531-b80e-cf2ffaf2f95b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:04.183430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9946d1b3-6d2a-4531-b80e-cf2ffaf2f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:04.288336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 675/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:04.579553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f01f7-e412-41a1-97f0-32607850602f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:04.581493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-879f01f7-e412-41a1-97f0-32607850602f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:04.585318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:04.767796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f01f7-e412-41a1-97f0-32607850602f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:04.858655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:04.918803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77827f6e-24e5-4cda-bc02-6e5d7784d7ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:04.920030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77827f6e-24e5-4cda-bc02-6e5d7784d7ea service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:04.920030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77827f6e-24e5-4cda-bc02-6e5d7784d7ea service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:04.920030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77827f6e-24e5-4cda-bc02-6e5d7784d7ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:04.921835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-77827f6e-24e5-4cda-bc02-6e5d7784d7ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:05.036383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77827f6e-24e5-4cda-bc02-6e5d7784d7ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:05.038549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77827f6e-24e5-4cda-bc02-6e5d7784d7ea service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:05.063762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 704/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:07.386292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742850ce-d0b3-489a-b83f-2665370c3429 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.386292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742850ce-d0b3-489a-b83f-2665370c3429 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.386292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742850ce-d0b3-489a-b83f-2665370c3429 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.386930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 676/1380] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:07.394041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d0d2a-f599-439d-aa9a-3dab13b1c157 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.394536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d0d2a-f599-439d-aa9a-3dab13b1c157 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.394647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d0d2a-f599-439d-aa9a-3dab13b1c157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.490949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d96d0d2a-f599-439d-aa9a-3dab13b1c157 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:07.492921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d96d0d2a-f599-439d-aa9a-3dab13b1c157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:07.539682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:07.612457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 705/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:07.906735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742850ce-d0b3-489a-b83f-2665370c3429 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.908571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-742850ce-d0b3-489a-b83f-2665370c3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:08.002912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:08.113453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742850ce-d0b3-489a-b83f-2665370c3429 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:08.236919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7bc8c-8dea-41f5-a538-cf7d7162c81a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:08.237543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bc8c-8dea-41f5-a538-cf7d7162c81a service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:08.237859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bc8c-8dea-41f5-a538-cf7d7162c81a service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:08.238133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bc8c-8dea-41f5-a538-cf7d7162c81a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:08.240164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7da7bc8c-8dea-41f5-a538-cf7d7162c81a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:08.261378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:08.351973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7bc8c-8dea-41f5-a538-cf7d7162c81a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:08.354282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7bc8c-8dea-41f5-a538-cf7d7162c81a service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:08.381883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 677/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:10.690806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf84552-ffb7-4f6b-9349-06493fad27b8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:10.691441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf84552-ffb7-4f6b-9349-06493fad27b8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:10.691441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf84552-ffb7-4f6b-9349-06493fad27b8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:10.692815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 706/1383] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:10.699456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30639cdf-4acf-46ea-8043-377e13036dcc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:10.700023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30639cdf-4acf-46ea-8043-377e13036dcc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:10.700140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30639cdf-4acf-46ea-8043-377e13036dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:10.775311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-30639cdf-4acf-46ea-8043-377e13036dcc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:10.776608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-30639cdf-4acf-46ea-8043-377e13036dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:10.880123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:10.888136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 678/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:11.183496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf84552-ffb7-4f6b-9349-06493fad27b8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:11.185014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-adf84552-ffb7-4f6b-9349-06493fad27b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:11.406132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf84552-ffb7-4f6b-9349-06493fad27b8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:11.434435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:11.582359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96a0e0-fdb9-46e1-95d3-abb138117bf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:11.582900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a0e0-fdb9-46e1-95d3-abb138117bf4 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:11.583279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a0e0-fdb9-46e1-95d3-abb138117bf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:11.583542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a0e0-fdb9-46e1-95d3-abb138117bf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:11.585492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2e96a0e0-fdb9-46e1-95d3-abb138117bf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:11.698933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96a0e0-fdb9-46e1-95d3-abb138117bf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:11.700772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96a0e0-fdb9-46e1-95d3-abb138117bf4 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:11.727438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 707/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 12:59:14.091699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e7c2-c5aa-4488-8416-f199d2e77802 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.092474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e7c2-c5aa-4488-8416-f199d2e77802 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.092474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e7c2-c5aa-4488-8416-f199d2e77802 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.093892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 679/1386] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:14.099556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a21b-a763-4b3a-b7ab-8762ad1fc647 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.100198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a21b-a763-4b3a-b7ab-8762ad1fc647 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.100198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a21b-a763-4b3a-b7ab-8762ad1fc647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.137271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:14.170339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ac15a21b-a763-4b3a-b7ab-8762ad1fc647 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:14.171569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ac15a21b-a763-4b3a-b7ab-8762ad1fc647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:14.273541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 708/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:14.576724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e7c2-c5aa-4488-8416-f199d2e77802 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:14.578300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ee73e7c2-c5aa-4488-8416-f199d2e77802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:14.739451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e7c2-c5aa-4488-8416-f199d2e77802 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:14.823986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:14.884255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae2db4-e021-4585-8dd4-5e758af7777e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:14.884859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae2db4-e021-4585-8dd4-5e758af7777e service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.885156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae2db4-e021-4585-8dd4-5e758af7777e service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.885412 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae2db4-e021-4585-8dd4-5e758af7777e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.887326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-85ae2db4-e021-4585-8dd4-5e758af7777e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:14.982398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae2db4-e021-4585-8dd4-5e758af7777e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:14.984172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae2db4-e021-4585-8dd4-5e758af7777e service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:14.986980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c078a7-73ea-4ae0-8acc-fdc8dfc559a5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.987271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c078a7-73ea-4ae0-8acc-fdc8dfc559a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.987544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c078a7-73ea-4ae0-8acc-fdc8dfc559a5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.988913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 709/1388] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:14.994633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3bf9e-e4ba-4ef7-84ef-4d8c41f72403 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.994914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3bf9e-e4ba-4ef7-84ef-4d8c41f72403 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.995219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3bf9e-e4ba-4ef7-84ef-4d8c41f72403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.003755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 680/1389] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:15.067214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-23e3bf9e-e4ba-4ef7-84ef-4d8c41f72403 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:15.069175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-23e3bf9e-e4ba-4ef7-84ef-4d8c41f72403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:15.136197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:15.189013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 710/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:15.502205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec521c-53d5-4aa4-93ec-d0893019b173 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.502722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec521c-53d5-4aa4-93ec-d0893019b173 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.502722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec521c-53d5-4aa4-93ec-d0893019b173 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.504332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 681/1391] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 12:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:15.510556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4407a4-ea85-43f5-a19c-f126764877d3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.510836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4407a4-ea85-43f5-a19c-f126764877d3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.511134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4407a4-ea85-43f5-a19c-f126764877d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.596490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9d4407a4-ea85-43f5-a19c-f126764877d3 None None] MFA Rules not processed for user `2aba60973c5e4aa7b0b680acb24d0431`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:15.597710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9d4407a4-ea85-43f5-a19c-f126764877d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:15.678769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 711/1392] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 12:59:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:15.688020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec521c-53d5-4aa4-93ec-d0893019b173 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:15.689529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5eec521c-53d5-4aa4-93ec-d0893019b173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:15.786421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:15.810893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec521c-53d5-4aa4-93ec-d0893019b173 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:15.891064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d752c-7214-4177-817a-42d7e74b3195 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:15.891739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d752c-7214-4177-817a-42d7e74b3195 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.892023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d752c-7214-4177-817a-42d7e74b3195 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.892314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d752c-7214-4177-817a-42d7e74b3195 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.894620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f70d752c-7214-4177-817a-42d7e74b3195 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:16.016042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d752c-7214-4177-817a-42d7e74b3195 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:16.017505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d752c-7214-4177-817a-42d7e74b3195 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:16.036776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 682/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:16.797770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb157f8-7042-4111-871c-39945ccc75d9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.798431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb157f8-7042-4111-871c-39945ccc75d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.798431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb157f8-7042-4111-871c-39945ccc75d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.873544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1bb157f8-7042-4111-871c-39945ccc75d9 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:16.874800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1bb157f8-7042-4111-871c-39945ccc75d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:16.877092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:16.971294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 712/1394] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 12:59:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:17.015700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920595e2-406b-4c9f-a5c8-3edf1a5014e5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.016256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920595e2-406b-4c9f-a5c8-3edf1a5014e5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.016256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920595e2-406b-4c9f-a5c8-3edf1a5014e5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.017609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 683/1395] 10.222.0.17 () {54 vars in 943 bytes} [Wed Sep 7 12:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:17.025333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146319d-c22b-4c25-8c69-acdc3c902981 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.025641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146319d-c22b-4c25-8c69-acdc3c902981 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.025944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146319d-c22b-4c25-8c69-acdc3c902981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.107387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6146319d-c22b-4c25-8c69-acdc3c902981 None None] MFA Rules not processed for user `30f70cdf08d049668d16007e4ca44654`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:17.108628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6146319d-c22b-4c25-8c69-acdc3c902981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:17.197265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:17.229387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 713/1396] 10.222.0.17 () {58 vars in 1040 bytes} [Wed Sep 7 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:17.234023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d02bf-b99c-4227-ac92-fc2065b74b63 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.234336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d02bf-b99c-4227-ac92-fc2065b74b63 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.234617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d02bf-b99c-4227-ac92-fc2065b74b63 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.236013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 684/1397] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:17.238997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146319d-c22b-4c25-8c69-acdc3c902981 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.241402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.241706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.241990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.249955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146319d-c22b-4c25-8c69-acdc3c902981 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.257068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:17.312307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:17.313565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:17.366883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069be1d-0733-4127-9e03-6e73b4f7d691 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.367495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069be1d-0733-4127-9e03-6e73b4f7d691 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.367765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069be1d-0733-4127-9e03-6e73b4f7d691 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.368020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069be1d-0733-4127-9e03-6e73b4f7d691 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.369687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5069be1d-0733-4127-9e03-6e73b4f7d691 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:17.415584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 685/1398] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 12:59:17.465628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069be1d-0733-4127-9e03-6e73b4f7d691 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:17.467294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069be1d-0733-4127-9e03-6e73b4f7d691 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:17.477098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 714/1399] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 12:59:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 12:59:17.702856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.704133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:17.821281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:17.856146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e04a54-1af6-49f9-9ae3-fda13d401056 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.991977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7fb51-0da4-4d0e-937f-bec250f74190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.992601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7fb51-0da4-4d0e-937f-bec250f74190 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.992884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7fb51-0da4-4d0e-937f-bec250f74190 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.993162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7fb51-0da4-4d0e-937f-bec250f74190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.996028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d8b7fb51-0da4-4d0e-937f-bec250f74190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:18.096025 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:18.101862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7fb51-0da4-4d0e-937f-bec250f74190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:18.103719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7fb51-0da4-4d0e-937f-bec250f74190 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:18.124970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 686/1400] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:20.415234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846030f-11ca-4da0-ae0b-8ae59a8d663d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:20.416185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846030f-11ca-4da0-ae0b-8ae59a8d663d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:20.416185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846030f-11ca-4da0-ae0b-8ae59a8d663d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:20.417278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 715/1401] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:20.424258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69a2a-7fa5-4740-8b56-4ec6a96f4f64 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:20.425108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69a2a-7fa5-4740-8b56-4ec6a96f4f64 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:20.425108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69a2a-7fa5-4740-8b56-4ec6a96f4f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:20.528866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c7a69a2a-7fa5-4740-8b56-4ec6a96f4f64 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:20.530446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c7a69a2a-7fa5-4740-8b56-4ec6a96f4f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:20.663211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 687/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:20.964576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846030f-11ca-4da0-ae0b-8ae59a8d663d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:20.965963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b846030f-11ca-4da0-ae0b-8ae59a8d663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:20.969846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:21.154526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846030f-11ca-4da0-ae0b-8ae59a8d663d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:21.288920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:21.345546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca6fd-84e1-4fa8-adae-6ce46816e847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:21.346182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca6fd-84e1-4fa8-adae-6ce46816e847 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:21.346473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca6fd-84e1-4fa8-adae-6ce46816e847 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:21.346739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca6fd-84e1-4fa8-adae-6ce46816e847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:21.348451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-940ca6fd-84e1-4fa8-adae-6ce46816e847 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:21.455786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ca6fd-84e1-4fa8-adae-6ce46816e847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:21.457400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ca6fd-84e1-4fa8-adae-6ce46816e847 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:21.477304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 716/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:23.761260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4721c-89af-4131-a9be-31e8d578a7ce None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.761931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4721c-89af-4131-a9be-31e8d578a7ce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.761931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4721c-89af-4131-a9be-31e8d578a7ce None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.763837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 688/1404] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:23.769595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ceb5f-61bc-43c2-8962-7e0249f0ac21 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.770132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ceb5f-61bc-43c2-8962-7e0249f0ac21 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.770223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ceb5f-61bc-43c2-8962-7e0249f0ac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.857347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e35ceb5f-61bc-43c2-8962-7e0249f0ac21 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:23.858730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e35ceb5f-61bc-43c2-8962-7e0249f0ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:23.909784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:23.984977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 717/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:24.276743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4721c-89af-4131-a9be-31e8d578a7ce None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:24.278624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8cd4721c-89af-4131-a9be-31e8d578a7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:24.360499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:24.461544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4721c-89af-4131-a9be-31e8d578a7ce None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:24.678332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02f276-0a50-42a7-8f6a-49140630892f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:24.678950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02f276-0a50-42a7-8f6a-49140630892f service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:24.679261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02f276-0a50-42a7-8f6a-49140630892f service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:24.679521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02f276-0a50-42a7-8f6a-49140630892f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:24.684476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-af02f276-0a50-42a7-8f6a-49140630892f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:24.752793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:24.826916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02f276-0a50-42a7-8f6a-49140630892f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:24.829471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02f276-0a50-42a7-8f6a-49140630892f service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:24.849731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 689/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:27.163444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0c477-6920-468d-8c39-61e040f9e9a8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.163999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0c477-6920-468d-8c39-61e040f9e9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.163999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0c477-6920-468d-8c39-61e040f9e9a8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.165243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 718/1407] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 12:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:27.170874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e583-5f8a-4bb8-963b-91652989d2f3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.171676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e583-5f8a-4bb8-963b-91652989d2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.171676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757e583-5f8a-4bb8-963b-91652989d2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.254679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3757e583-5f8a-4bb8-963b-91652989d2f3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:27.256042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3757e583-5f8a-4bb8-963b-91652989d2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:27.387230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 690/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:27.692844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0c477-6920-468d-8c39-61e040f9e9a8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.694517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5cb0c477-6920-468d-8c39-61e040f9e9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:27.893707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0c477-6920-468d-8c39-61e040f9e9a8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.921033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:28.056910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3aa4e-2eb0-4ce7-841c-6e3f6823e0d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:28.058019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3aa4e-2eb0-4ce7-841c-6e3f6823e0d9 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:28.058589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3aa4e-2eb0-4ce7-841c-6e3f6823e0d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:28.058995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3aa4e-2eb0-4ce7-841c-6e3f6823e0d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:28.062020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f1f3aa4e-2eb0-4ce7-841c-6e3f6823e0d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:28.183657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3aa4e-2eb0-4ce7-841c-6e3f6823e0d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:28.185080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3aa4e-2eb0-4ce7-841c-6e3f6823e0d9 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:28.207965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 719/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:37.384212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:38.204569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:41.371674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc620c-0dbb-4ca3-bb97-43bd5969815a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:41.372295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc620c-0dbb-4ca3-bb97-43bd5969815a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:41.372295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc620c-0dbb-4ca3-bb97-43bd5969815a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:41.373582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 691/1410] 10.222.0.17 () {54 vars in 918 bytes} [Wed Sep 7 12:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:41.379309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cfe54-27b7-469f-9c72-92f16a62d9d8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:41.380629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cfe54-27b7-469f-9c72-92f16a62d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:41.380629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cfe54-27b7-469f-9c72-92f16a62d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:41.468375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-473cfe54-27b7-469f-9c72-92f16a62d9d8 None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:41.470466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-473cfe54-27b7-469f-9c72-92f16a62d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:41.591129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 720/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Sep 7 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:41.606155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247e13-a715-4ef6-9ded-de2a0ed2207e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:41.606444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247e13-a715-4ef6-9ded-de2a0ed2207e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:41.606720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247e13-a715-4ef6-9ded-de2a0ed2207e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:41.608061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 692/1412] 10.222.0.17 () {54 vars in 918 bytes} [Wed Sep 7 12:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:41.613310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55017903-85a6-4d71-b81c-5e36a9e10ea8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:41.613597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55017903-85a6-4d71-b81c-5e36a9e10ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:41.613896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55017903-85a6-4d71-b81c-5e36a9e10ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:41.693850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:41.697936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-55017903-85a6-4d71-b81c-5e36a9e10ea8 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:41.699629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-55017903-85a6-4d71-b81c-5e36a9e10ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:41.830018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 721/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Sep 7 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:44.534679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436f83-e17b-4572-8922-b64b65194a77 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.535263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436f83-e17b-4572-8922-b64b65194a77 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.536683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a436f83-e17b-4572-8922-b64b65194a77 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.538250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 693/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 12:59:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:44.544395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199272-cc0f-4ae0-828d-7fd62526577d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.544931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199272-cc0f-4ae0-828d-7fd62526577d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.545039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b199272-cc0f-4ae0-828d-7fd62526577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.623595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7b199272-cc0f-4ae0-828d-7fd62526577d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:44.624863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7b199272-cc0f-4ae0-828d-7fd62526577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:44.685148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:44.733945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 722/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 12:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:45.029841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436f83-e17b-4572-8922-b64b65194a77 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:45.031444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3a436f83-e17b-4572-8922-b64b65194a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:45.188109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a436f83-e17b-4572-8922-b64b65194a77 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:45.266713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:45.308579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be48afb-b0ea-485b-afb9-3442ff5e8c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:45.309198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48afb-b0ea-485b-afb9-3442ff5e8c8f service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:45.309475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48afb-b0ea-485b-afb9-3442ff5e8c8f service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:45.309797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48afb-b0ea-485b-afb9-3442ff5e8c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:45.311370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1be48afb-b0ea-485b-afb9-3442ff5e8c8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:45.406554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be48afb-b0ea-485b-afb9-3442ff5e8c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:45.408168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be48afb-b0ea-485b-afb9-3442ff5e8c8f service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:45.427231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 694/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:46.589652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ebc4-6b47-4fe2-bd88-5c71f8ac7fb0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:46.590319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ebc4-6b47-4fe2-bd88-5c71f8ac7fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:46.590319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ebc4-6b47-4fe2-bd88-5c71f8ac7fb0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:46.591573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 723/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:46.598294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54665bc-3902-4a59-91f6-165a0b8409ce None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:46.598843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54665bc-3902-4a59-91f6-165a0b8409ce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:46.598949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54665bc-3902-4a59-91f6-165a0b8409ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:46.680289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b54665bc-3902-4a59-91f6-165a0b8409ce None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:46.681419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b54665bc-3902-4a59-91f6-165a0b8409ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:46.710309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:46.785273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 695/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:46.935222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ebc4-6b47-4fe2-bd88-5c71f8ac7fb0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:46.936795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4eb4ebc4-6b47-4fe2-bd88-5c71f8ac7fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:47.098634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ebc4-6b47-4fe2-bd88-5c71f8ac7fb0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:47.143632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:47.223835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32d405-203e-44a8-a8c2-4c3381e1f9b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:47.224434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32d405-203e-44a8-a8c2-4c3381e1f9b0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:47.224824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32d405-203e-44a8-a8c2-4c3381e1f9b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:47.225098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32d405-203e-44a8-a8c2-4c3381e1f9b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:47.226776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5e32d405-203e-44a8-a8c2-4c3381e1f9b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:47.321861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32d405-203e-44a8-a8c2-4c3381e1f9b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:47.323476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32d405-203e-44a8-a8c2-4c3381e1f9b0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:47.342404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 724/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:48.494726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96406e-2b6b-4bda-a630-e2ca7fa9a1c9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.495333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96406e-2b6b-4bda-a630-e2ca7fa9a1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.495333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96406e-2b6b-4bda-a630-e2ca7fa9a1c9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.496634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 696/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:48.504052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d86fe8-400f-4b2c-8f34-f3975634a861 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.504632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d86fe8-400f-4b2c-8f34-f3975634a861 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.504632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d86fe8-400f-4b2c-8f34-f3975634a861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.565056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:48.590975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a4d86fe8-400f-4b2c-8f34-f3975634a861 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:48.592233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a4d86fe8-400f-4b2c-8f34-f3975634a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:48.698519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 725/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 12:59:48.850798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96406e-2b6b-4bda-a630-e2ca7fa9a1c9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:48.853271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cd96406e-2b6b-4bda-a630-e2ca7fa9a1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:49.027293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96406e-2b6b-4bda-a630-e2ca7fa9a1c9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:49.042231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:49.174759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cac65-835c-4f83-833a-b0480fa77148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:49.175364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cac65-835c-4f83-833a-b0480fa77148 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.175638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cac65-835c-4f83-833a-b0480fa77148 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.175910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cac65-835c-4f83-833a-b0480fa77148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.178098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d27cac65-835c-4f83-833a-b0480fa77148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:49.288281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cac65-835c-4f83-833a-b0480fa77148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:49.289908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cac65-835c-4f83-833a-b0480fa77148 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:49.312672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 697/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:50.482496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f20eaa-bb90-46f3-b9ce-57f4b5dd1754 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.483142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f20eaa-bb90-46f3-b9ce-57f4b5dd1754 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.483142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f20eaa-bb90-46f3-b9ce-57f4b5dd1754 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.484369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 726/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:50.491636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639260a7-588c-4988-a834-521b6f5635e6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.492264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639260a7-588c-4988-a834-521b6f5635e6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.492264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639260a7-588c-4988-a834-521b6f5635e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.513717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:50.576529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-639260a7-588c-4988-a834-521b6f5635e6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:50.577790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-639260a7-588c-4988-a834-521b6f5635e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:50.684229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 698/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:50.832510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f20eaa-bb90-46f3-b9ce-57f4b5dd1754 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:50.834294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12f20eaa-bb90-46f3-b9ce-57f4b5dd1754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:50.961315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:51.000754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f20eaa-bb90-46f3-b9ce-57f4b5dd1754 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:51.121274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01de0-bd17-4e1e-944b-bb15cef4adc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:51.121940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01de0-bd17-4e1e-944b-bb15cef4adc7 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:51.122198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01de0-bd17-4e1e-944b-bb15cef4adc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:51.122467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01de0-bd17-4e1e-944b-bb15cef4adc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:51.124095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-46f01de0-bd17-4e1e-944b-bb15cef4adc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:51.203428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:51.218127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01de0-bd17-4e1e-944b-bb15cef4adc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:51.219742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01de0-bd17-4e1e-944b-bb15cef4adc7 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:51.240685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 727/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:52.399833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27f4f9-0b05-4f0a-8f92-ecc0e07a2feb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:52.400451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27f4f9-0b05-4f0a-8f92-ecc0e07a2feb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:52.400451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27f4f9-0b05-4f0a-8f92-ecc0e07a2feb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:52.401784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 699/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:52.409087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426d60c-d5b9-459c-b793-189d6452cc3c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:52.409732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426d60c-d5b9-459c-b793-189d6452cc3c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:52.409732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426d60c-d5b9-459c-b793-189d6452cc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:52.523228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c426d60c-d5b9-459c-b793-189d6452cc3c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:52.524756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c426d60c-d5b9-459c-b793-189d6452cc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:52.629586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 728/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:52.781883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27f4f9-0b05-4f0a-8f92-ecc0e07a2feb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:52.783470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0c27f4f9-0b05-4f0a-8f92-ecc0e07a2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:52.872031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:52.937543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27f4f9-0b05-4f0a-8f92-ecc0e07a2feb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:53.063291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b647c1e-3b0a-45d1-b553-ac9adbe61000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:53.063902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b647c1e-3b0a-45d1-b553-ac9adbe61000 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.064182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b647c1e-3b0a-45d1-b553-ac9adbe61000 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.064479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b647c1e-3b0a-45d1-b553-ac9adbe61000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.066118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6b647c1e-3b0a-45d1-b553-ac9adbe61000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:53.120022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:53.163819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b647c1e-3b0a-45d1-b553-ac9adbe61000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:53.165322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b647c1e-3b0a-45d1-b553-ac9adbe61000 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:53.186591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 700/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:54.278925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4871e00-a37c-4b89-9bc5-715969b28f79 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:54.279544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4871e00-a37c-4b89-9bc5-715969b28f79 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:54.279544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4871e00-a37c-4b89-9bc5-715969b28f79 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:54.280894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 729/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:54.287891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aaa32-8e3c-4f6e-92b9-8e8701a0e688 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:54.288429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aaa32-8e3c-4f6e-92b9-8e8701a0e688 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:54.288429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aaa32-8e3c-4f6e-92b9-8e8701a0e688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:54.378243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a54aaa32-8e3c-4f6e-92b9-8e8701a0e688 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:54.379781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a54aaa32-8e3c-4f6e-92b9-8e8701a0e688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:54.494681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 701/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:54.653275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4871e00-a37c-4b89-9bc5-715969b28f79 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:54.654847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4871e00-a37c-4b89-9bc5-715969b28f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:54.706738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:54.811792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4871e00-a37c-4b89-9bc5-715969b28f79 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:54.935242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f250df-c2b6-471c-aee9-da66c216c1fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:54.935846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f250df-c2b6-471c-aee9-da66c216c1fc service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:54.936122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f250df-c2b6-471c-aee9-da66c216c1fc service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:54.936387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f250df-c2b6-471c-aee9-da66c216c1fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:54.938075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e3f250df-c2b6-471c-aee9-da66c216c1fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:54.945559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:55.037028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f250df-c2b6-471c-aee9-da66c216c1fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:55.038689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f250df-c2b6-471c-aee9-da66c216c1fc service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:55.057632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 730/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:56.136614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8a10d-47dd-4574-b5ca-80af234a4706 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:56.137223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8a10d-47dd-4574-b5ca-80af234a4706 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:56.137223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8a10d-47dd-4574-b5ca-80af234a4706 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:56.138555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 702/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:56.145594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64a81b-8dae-42ee-9509-3116fbe0a861 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:56.146179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64a81b-8dae-42ee-9509-3116fbe0a861 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:56.146179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64a81b-8dae-42ee-9509-3116fbe0a861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:56.229578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0e64a81b-8dae-42ee-9509-3116fbe0a861 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:56.230842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0e64a81b-8dae-42ee-9509-3116fbe0a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:56.304388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:56.334738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 731/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:56.484587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8a10d-47dd-4574-b5ca-80af234a4706 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:56.486207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d0d8a10d-47dd-4574-b5ca-80af234a4706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:56.504972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:56.649389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8a10d-47dd-4574-b5ca-80af234a4706 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:56.742310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:56.769800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a74c10-767f-4e2d-8837-b1b7e3c28a91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:56.770547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a74c10-767f-4e2d-8837-b1b7e3c28a91 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:56.770824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a74c10-767f-4e2d-8837-b1b7e3c28a91 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:56.771096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a74c10-767f-4e2d-8837-b1b7e3c28a91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:56.772700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-33a74c10-767f-4e2d-8837-b1b7e3c28a91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:56.870861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a74c10-767f-4e2d-8837-b1b7e3c28a91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:56.872530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a74c10-767f-4e2d-8837-b1b7e3c28a91 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:56.894190 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 703/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:57.979154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cbaf2-16cd-4be4-9797-5b0781ccbc30 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:57.979776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cbaf2-16cd-4be4-9797-5b0781ccbc30 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:57.979776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cbaf2-16cd-4be4-9797-5b0781ccbc30 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:57.981091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 732/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:57.989292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48f64-49fb-4a2a-95e0-9be5830ea26a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:57.989932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48f64-49fb-4a2a-95e0-9be5830ea26a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:57.990167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d48f64-49fb-4a2a-95e0-9be5830ea26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:58.084545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c0d48f64-49fb-4a2a-95e0-9be5830ea26a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:58.085842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c0d48f64-49fb-4a2a-95e0-9be5830ea26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:58.141189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:58.205788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 704/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 12:59:58.357202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cbaf2-16cd-4be4-9797-5b0781ccbc30 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:58.358727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8f4cbaf2-16cd-4be4-9797-5b0781ccbc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:58.524017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cbaf2-16cd-4be4-9797-5b0781ccbc30 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:58.586025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:58.654624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83df90-4c38-48d6-9cc5-eda289b95894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:58.655138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83df90-4c38-48d6-9cc5-eda289b95894 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:58.655413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83df90-4c38-48d6-9cc5-eda289b95894 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:58.655678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83df90-4c38-48d6-9cc5-eda289b95894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:58.657306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e83df90-4c38-48d6-9cc5-eda289b95894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:58.768588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83df90-4c38-48d6-9cc5-eda289b95894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:58.770454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83df90-4c38-48d6-9cc5-eda289b95894 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:58.792437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 733/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 12:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 12:59:59.892133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42ddb8-96b7-49cb-8128-441016e1aeba None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.892743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42ddb8-96b7-49cb-8128-441016e1aeba None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.892743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42ddb8-96b7-49cb-8128-441016e1aeba None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.894135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 705/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 12:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:59.901436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3508-c45a-45f7-bf55-bcd46875dd2d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.902018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3508-c45a-45f7-bf55-bcd46875dd2d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.902018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3508-c45a-45f7-bf55-bcd46875dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.984051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-670b3508-c45a-45f7-bf55-bcd46875dd2d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:59.985199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-670b3508-c45a-45f7-bf55-bcd46875dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:59.987348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:00.088378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 734/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 12:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:00:00.236461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42ddb8-96b7-49cb-8128-441016e1aeba None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:00.238068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aa42ddb8-96b7-49cb-8128-441016e1aeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:00.395854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42ddb8-96b7-49cb-8128-441016e1aeba None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:00.425935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:00.523333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73e5ff-ec40-436d-93e9-be6f7d629825 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:00.523858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e5ff-ec40-436d-93e9-be6f7d629825 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:00.524140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e5ff-ec40-436d-93e9-be6f7d629825 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:00.524415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73e5ff-ec40-436d-93e9-be6f7d629825 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.526074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9c73e5ff-ec40-436d-93e9-be6f7d629825 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:00.618546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73e5ff-ec40-436d-93e9-be6f7d629825 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:00.620209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73e5ff-ec40-436d-93e9-be6f7d629825 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:00.639339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 706/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:02.747215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ebaec-b026-4951-ac58-1bff9ec2826c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:02.747829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ebaec-b026-4951-ac58-1bff9ec2826c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:02.747829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ebaec-b026-4951-ac58-1bff9ec2826c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.749162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 735/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:02.754948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8d290-1a88-4526-8005-7df4d8e858cd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:02.755428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8d290-1a88-4526-8005-7df4d8e858cd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:02.755527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8d290-1a88-4526-8005-7df4d8e858cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.799376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:02.829932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ecf8d290-1a88-4526-8005-7df4d8e858cd None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:02.831224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecf8d290-1a88-4526-8005-7df4d8e858cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:02.935195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 707/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:03.216823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ebaec-b026-4951-ac58-1bff9ec2826c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.218459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1b4ebaec-b026-4951-ac58-1bff9ec2826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:03.331894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ebaec-b026-4951-ac58-1bff9ec2826c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:03.365520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:03.412171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc654a-463a-45a2-b0c3-e95052876c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:03.412794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc654a-463a-45a2-b0c3-e95052876c4e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:03.413071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc654a-463a-45a2-b0c3-e95052876c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:03.413339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc654a-463a-45a2-b0c3-e95052876c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:03.414991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bacc654a-463a-45a2-b0c3-e95052876c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:03.535978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc654a-463a-45a2-b0c3-e95052876c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:03.537394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc654a-463a-45a2-b0c3-e95052876c4e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:03.558259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 736/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:05.891259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565c444-a2d3-4276-8850-2fc2fc193025 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:05.891836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565c444-a2d3-4276-8850-2fc2fc193025 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:05.891836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565c444-a2d3-4276-8850-2fc2fc193025 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:05.893152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 708/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:00:05.900086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ff2f-87a2-4896-92c8-40f62beb4b2e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:05.900617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ff2f-87a2-4896-92c8-40f62beb4b2e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:05.900617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ff2f-87a2-4896-92c8-40f62beb4b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:05.976007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5311ff2f-87a2-4896-92c8-40f62beb4b2e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:05.977224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5311ff2f-87a2-4896-92c8-40f62beb4b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:05.990506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:06.078710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 737/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:06.361084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565c444-a2d3-4276-8850-2fc2fc193025 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:06.362868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3565c444-a2d3-4276-8850-2fc2fc193025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:06.475407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565c444-a2d3-4276-8850-2fc2fc193025 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:06.489174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:06.557561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d513d-0a29-4a20-9278-4917558aa2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:06.558210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d513d-0a29-4a20-9278-4917558aa2ed service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:06.558521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d513d-0a29-4a20-9278-4917558aa2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:06.558788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d513d-0a29-4a20-9278-4917558aa2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:06.560426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-239d513d-0a29-4a20-9278-4917558aa2ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:06.679499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d513d-0a29-4a20-9278-4917558aa2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:06.680822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d513d-0a29-4a20-9278-4917558aa2ed service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:06.699866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 709/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:09.249426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620e85-8651-4ddd-8185-93226ccf0eac None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.250025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620e85-8651-4ddd-8185-93226ccf0eac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.250025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620e85-8651-4ddd-8185-93226ccf0eac None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.251355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 738/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:09.257249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308921c-880c-418b-add4-9b2aa77e6092 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.257770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308921c-880c-418b-add4-9b2aa77e6092 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.257885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308921c-880c-418b-add4-9b2aa77e6092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.315636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:09.341660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6308921c-880c-418b-add4-9b2aa77e6092 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:09.342928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6308921c-880c-418b-add4-9b2aa77e6092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:09.453172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 710/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:09.749974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620e85-8651-4ddd-8185-93226ccf0eac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.751892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7620e85-8651-4ddd-8185-93226ccf0eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:09.866921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620e85-8651-4ddd-8185-93226ccf0eac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.914162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:09.956627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f014f2-02d9-4d6b-8dc2-25e74497b95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.957260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f014f2-02d9-4d6b-8dc2-25e74497b95c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.957542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f014f2-02d9-4d6b-8dc2-25e74497b95c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.957856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f014f2-02d9-4d6b-8dc2-25e74497b95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.961934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-82f014f2-02d9-4d6b-8dc2-25e74497b95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:10.228767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f014f2-02d9-4d6b-8dc2-25e74497b95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:10.230288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f014f2-02d9-4d6b-8dc2-25e74497b95c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:10.251817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 739/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:12.634202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abb98-6a42-4960-8b62-7087dfd9981c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.634792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abb98-6a42-4960-8b62-7087dfd9981c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.634792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abb98-6a42-4960-8b62-7087dfd9981c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:12.636005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 711/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:12.641699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f696-7197-49f8-b807-1ad83538cd7e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.642180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f696-7197-49f8-b807-1ad83538cd7e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.642300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f696-7197-49f8-b807-1ad83538cd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:12.713327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4704f696-7197-49f8-b807-1ad83538cd7e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:12.714584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4704f696-7197-49f8-b807-1ad83538cd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:12.736943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:12.818860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 740/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:13.100613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7abb98-6a42-4960-8b62-7087dfd9981c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:13.102240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7e7abb98-6a42-4960-8b62-7087dfd9981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:13.220041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7abb98-6a42-4960-8b62-7087dfd9981c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:13.241891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:13.303372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37a3e-b49d-47bc-8e85-798b81d0c41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:13.303999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37a3e-b49d-47bc-8e85-798b81d0c41a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:13.304282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37a3e-b49d-47bc-8e85-798b81d0c41a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:13.304564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37a3e-b49d-47bc-8e85-798b81d0c41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:13.306540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9ce37a3e-b49d-47bc-8e85-798b81d0c41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:13.436705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce37a3e-b49d-47bc-8e85-798b81d0c41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:13.438750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce37a3e-b49d-47bc-8e85-798b81d0c41a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:13.462851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 712/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:15.907509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dd421-1411-47f0-935c-e047e73ed5d4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.910519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dd421-1411-47f0-935c-e047e73ed5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.910519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dd421-1411-47f0-935c-e047e73ed5d4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.910519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 741/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:15.916431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a758-99f9-48a2-bfcd-86eee7335bba None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.916932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a758-99f9-48a2-bfcd-86eee7335bba None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.917027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a758-99f9-48a2-bfcd-86eee7335bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.988644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:15.991997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9b69a758-99f9-48a2-bfcd-86eee7335bba None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:15.993963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9b69a758-99f9-48a2-bfcd-86eee7335bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:16.103762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 713/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:16.387141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dd421-1411-47f0-935c-e047e73ed5d4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.388687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa6dd421-1411-47f0-935c-e047e73ed5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:16.509958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dd421-1411-47f0-935c-e047e73ed5d4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.561134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:16.589288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24f0dc-7da7-4fd3-8ca2-903ef3b42f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.589933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24f0dc-7da7-4fd3-8ca2-903ef3b42f2e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.590211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24f0dc-7da7-4fd3-8ca2-903ef3b42f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.590477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24f0dc-7da7-4fd3-8ca2-903ef3b42f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.592372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4a24f0dc-7da7-4fd3-8ca2-903ef3b42f2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:16.724151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24f0dc-7da7-4fd3-8ca2-903ef3b42f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.725650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24f0dc-7da7-4fd3-8ca2-903ef3b42f2e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.752412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 742/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:19.139312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f67492-d757-4b24-8ed8-ca229a3a061a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:19.139910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f67492-d757-4b24-8ed8-ca229a3a061a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:19.140131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f67492-d757-4b24-8ed8-ca229a3a061a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:19.142554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 714/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:19.149564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49934e8-71b9-4319-a12e-a71ab243b25a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:19.150611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49934e8-71b9-4319-a12e-a71ab243b25a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:19.150611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49934e8-71b9-4319-a12e-a71ab243b25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:19.226444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b49934e8-71b9-4319-a12e-a71ab243b25a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:19.227681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b49934e8-71b9-4319-a12e-a71ab243b25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:19.258948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:19.333859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 743/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:19.621925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f67492-d757-4b24-8ed8-ca229a3a061a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:19.623489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-34f67492-d757-4b24-8ed8-ca229a3a061a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:19.746842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f67492-d757-4b24-8ed8-ca229a3a061a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:19.778971 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:19.827457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938421a6-52bd-4723-b910-cc49ec769762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:19.828070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938421a6-52bd-4723-b910-cc49ec769762 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:19.828347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938421a6-52bd-4723-b910-cc49ec769762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:19.828619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938421a6-52bd-4723-b910-cc49ec769762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:19.830442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-938421a6-52bd-4723-b910-cc49ec769762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:19.953646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938421a6-52bd-4723-b910-cc49ec769762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:19.955337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938421a6-52bd-4723-b910-cc49ec769762 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:19.975755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 715/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:22.310953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fbccf-4c55-4272-8379-a92bad4fbe92 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.311663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fbccf-4c55-4272-8379-a92bad4fbe92 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.311663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fbccf-4c55-4272-8379-a92bad4fbe92 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.313046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 744/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:22.319634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c455-4771-4d30-8355-ea24362bae5c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.320249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c455-4771-4d30-8355-ea24362bae5c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.320249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c455-4771-4d30-8355-ea24362bae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.397591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3378c455-4771-4d30-8355-ea24362bae5c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:22.398906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3378c455-4771-4d30-8355-ea24362bae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:22.408312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:22.517841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 716/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:22.807959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fbccf-4c55-4272-8379-a92bad4fbe92 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:22.809523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7d1fbccf-4c55-4272-8379-a92bad4fbe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:22.939044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fbccf-4c55-4272-8379-a92bad4fbe92 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:23.001854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:23.021058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3889011-7cce-4f85-9416-838bc5488919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:23.021686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3889011-7cce-4f85-9416-838bc5488919 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.021969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3889011-7cce-4f85-9416-838bc5488919 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.022239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3889011-7cce-4f85-9416-838bc5488919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.023850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d3889011-7cce-4f85-9416-838bc5488919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:23.146596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3889011-7cce-4f85-9416-838bc5488919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:23.148009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3889011-7cce-4f85-9416-838bc5488919 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:23.170062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 745/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:25.428654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a5f8b-36ae-4bf4-ba07-86be3472aa48 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:25.429229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a5f8b-36ae-4bf4-ba07-86be3472aa48 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:25.429229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a5f8b-36ae-4bf4-ba07-86be3472aa48 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:25.430570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 717/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:25.436233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813f5e-4f31-456e-903b-74a50dcf5b9d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:25.436734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813f5e-4f31-456e-903b-74a50dcf5b9d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:25.436734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813f5e-4f31-456e-903b-74a50dcf5b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:25.510030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-76813f5e-4f31-456e-903b-74a50dcf5b9d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:25.511261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-76813f5e-4f31-456e-903b-74a50dcf5b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:25.552661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:25.623147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 746/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:00:25.912621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a5f8b-36ae-4bf4-ba07-86be3472aa48 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:25.914262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-db7a5f8b-36ae-4bf4-ba07-86be3472aa48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:26.036876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a5f8b-36ae-4bf4-ba07-86be3472aa48 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:26.081704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:26.131864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c5856-3020-442b-8217-cb6c804cea99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:26.132551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5856-3020-442b-8217-cb6c804cea99 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:26.132836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5856-3020-442b-8217-cb6c804cea99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:26.133109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5856-3020-442b-8217-cb6c804cea99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:26.134858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3c7c5856-3020-442b-8217-cb6c804cea99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:26.265550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c5856-3020-442b-8217-cb6c804cea99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:26.267630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c5856-3020-442b-8217-cb6c804cea99 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:26.290161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 718/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:28.623866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359b03-7cfa-4d19-8569-66b6915063e8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.624434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359b03-7cfa-4d19-8569-66b6915063e8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.624434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359b03-7cfa-4d19-8569-66b6915063e8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.626472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 747/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:28.633080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135da48-fea8-41ab-98d8-cd87706023c8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.633653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135da48-fea8-41ab-98d8-cd87706023c8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.633653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135da48-fea8-41ab-98d8-cd87706023c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.710831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a135da48-fea8-41ab-98d8-cd87706023c8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:28.712077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a135da48-fea8-41ab-98d8-cd87706023c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:28.733812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:28.826654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 719/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:29.123421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1359b03-7cfa-4d19-8569-66b6915063e8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:29.125077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b1359b03-7cfa-4d19-8569-66b6915063e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:29.249986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1359b03-7cfa-4d19-8569-66b6915063e8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:29.331463 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.340399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a3b41-82ec-4a67-8ca5-fc3631b20c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.341160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a3b41-82ec-4a67-8ca5-fc3631b20c47 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.341627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a3b41-82ec-4a67-8ca5-fc3631b20c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.342026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a3b41-82ec-4a67-8ca5-fc3631b20c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.344029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a70a3b41-82ec-4a67-8ca5-fc3631b20c47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:29.471609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a3b41-82ec-4a67-8ca5-fc3631b20c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:29.473068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a3b41-82ec-4a67-8ca5-fc3631b20c47 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:29.492929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 748/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:31.866044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4379c-6401-456b-ab80-6355c26f0f7a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.866661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4379c-6401-456b-ab80-6355c26f0f7a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.866770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4379c-6401-456b-ab80-6355c26f0f7a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.868200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 720/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:31.874870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079fd7-f39b-4a3b-9327-bf7e9de7908b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.875461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079fd7-f39b-4a3b-9327-bf7e9de7908b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.875461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079fd7-f39b-4a3b-9327-bf7e9de7908b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.974108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-db079fd7-f39b-4a3b-9327-bf7e9de7908b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:31.975548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db079fd7-f39b-4a3b-9327-bf7e9de7908b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:32.027845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:32.115095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 749/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:32.404976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4379c-6401-456b-ab80-6355c26f0f7a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.406985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c9b4379c-6401-456b-ab80-6355c26f0f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:32.545575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4379c-6401-456b-ab80-6355c26f0f7a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:32.607897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:32.647376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f783c-84fa-4299-acf4-ed6efc80c5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.648085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f783c-84fa-4299-acf4-ed6efc80c5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.648522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f783c-84fa-4299-acf4-ed6efc80c5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.648877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f783c-84fa-4299-acf4-ed6efc80c5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.650850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2a5f783c-84fa-4299-acf4-ed6efc80c5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:32.786875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f783c-84fa-4299-acf4-ed6efc80c5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.788322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f783c-84fa-4299-acf4-ed6efc80c5b2 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.809296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 721/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:00:42.111298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=22 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:42.806879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=66 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:53.345110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b06443-2b3d-4422-9d54-e79c07d1da78 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.345825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b06443-2b3d-4422-9d54-e79c07d1da78 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.345825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b06443-2b3d-4422-9d54-e79c07d1da78 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.346937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 750/1471] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 13:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:53.353550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afd532-f84e-4e5f-9f3a-05240e1e88f2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.354147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afd532-f84e-4e5f-9f3a-05240e1e88f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.354258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afd532-f84e-4e5f-9f3a-05240e1e88f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.431020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b7afd532-f84e-4e5f-9f3a-05240e1e88f2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:53.432261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b7afd532-f84e-4e5f-9f3a-05240e1e88f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:53.543186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 722/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:53.847114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75740191-8b4f-4015-8ce6-a8702955826c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.847435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75740191-8b4f-4015-8ce6-a8702955826c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.847687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75740191-8b4f-4015-8ce6-a8702955826c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.849048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 751/1473] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 13:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:53.854789 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0be90-1037-49df-a060-73f114063708 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.855114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0be90-1037-49df-a060-73f114063708 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.855443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0be90-1037-49df-a060-73f114063708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.977394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-56c0be90-1037-49df-a060-73f114063708 None None] MFA Rules not processed for user `2aba60973c5e4aa7b0b680acb24d0431`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:53.979474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-56c0be90-1037-49df-a060-73f114063708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:54.023208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:54.114817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 723/1474] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 13:00:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:54.122895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75740191-8b4f-4015-8ce6-a8702955826c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:54.124751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-75740191-8b4f-4015-8ce6-a8702955826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:54.238947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75740191-8b4f-4015-8ce6-a8702955826c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:54.319526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341668e4-0b8a-42d9-b774-4731b4e355e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:54.320142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341668e4-0b8a-42d9-b774-4731b4e355e5 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:54.320420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341668e4-0b8a-42d9-b774-4731b4e355e5 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:54.320702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341668e4-0b8a-42d9-b774-4731b4e355e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:54.322501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-341668e4-0b8a-42d9-b774-4731b4e355e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:54.381064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:54.444649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341668e4-0b8a-42d9-b774-4731b4e355e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:54.446155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341668e4-0b8a-42d9-b774-4731b4e355e5 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:54.466207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 752/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:00:56.771588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b87a5-1555-4e49-bce7-0e18ce2520a7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:56.772187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b87a5-1555-4e49-bce7-0e18ce2520a7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:56.772187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b87a5-1555-4e49-bce7-0e18ce2520a7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:56.773509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 724/1476] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 13:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:00:56.780470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1177-cbe8-4883-8d73-589f41d8b43b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:56.782211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1177-cbe8-4883-8d73-589f41d8b43b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:56.782211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1177-cbe8-4883-8d73-589f41d8b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:56.871322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1cba1177-cbe8-4883-8d73-589f41d8b43b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:56.873059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1cba1177-cbe8-4883-8d73-589f41d8b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:56.995786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 753/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:00:57.268754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b87a5-1555-4e49-bce7-0e18ce2520a7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:57.270341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0c3b87a5-1555-4e49-bce7-0e18ce2520a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:57.420097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b87a5-1555-4e49-bce7-0e18ce2520a7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:57.488362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:57.539766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71578899-3110-45ba-a0d9-e64a9970af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:57.540265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578899-3110-45ba-a0d9-e64a9970af3a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:57.540540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578899-3110-45ba-a0d9-e64a9970af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:57.540807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578899-3110-45ba-a0d9-e64a9970af3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:57.541439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71578899-3110-45ba-a0d9-e64a9970af3a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:57.543317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-71578899-3110-45ba-a0d9-e64a9970af3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:00:57.543317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:00:57.543317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:00:57.544576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71578899-3110-45ba-a0d9-e64a9970af3a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:57.564117 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 725/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:57 2022] GET /identity/v3/projects => generated 2553 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:00:59.959225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a278a-b9b3-4248-82d9-0237c9b95765 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.959965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a278a-b9b3-4248-82d9-0237c9b95765 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.959965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a278a-b9b3-4248-82d9-0237c9b95765 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.961141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 754/1479] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 13:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:59.968335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9f59b-e73b-4b1e-a3ee-fc3cf8377ba6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.969023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9f59b-e73b-4b1e-a3ee-fc3cf8377ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.969136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9f59b-e73b-4b1e-a3ee-fc3cf8377ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.054795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-eaf9f59b-e73b-4b1e-a3ee-fc3cf8377ba6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:00.056059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-eaf9f59b-e73b-4b1e-a3ee-fc3cf8377ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:00.164279 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 726/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:01:00.465130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a278a-b9b3-4248-82d9-0237c9b95765 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.467813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b78a278a-b9b3-4248-82d9-0237c9b95765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:00.478964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:00.633698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a278a-b9b3-4248-82d9-0237c9b95765 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:00.720101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:00.756314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5848f03-4ebf-4be0-a738-c4e9e8e10c8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:00.756924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5848f03-4ebf-4be0-a738-c4e9e8e10c8a service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:00.757203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5848f03-4ebf-4be0-a738-c4e9e8e10c8a service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:00.757469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5848f03-4ebf-4be0-a738-c4e9e8e10c8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.759108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5848f03-4ebf-4be0-a738-c4e9e8e10c8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:00.857031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5848f03-4ebf-4be0-a738-c4e9e8e10c8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:00.858675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5848f03-4ebf-4be0-a738-c4e9e8e10c8a service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:00.879172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 755/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:01:03.291954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1227b-5244-444f-b04e-6f4c9dfc0c40 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:03.292535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1227b-5244-444f-b04e-6f4c9dfc0c40 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:03.292535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1227b-5244-444f-b04e-6f4c9dfc0c40 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:03.294648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 727/1482] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 13:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:03.301086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440a05-6621-4317-83c5-e38bc4cbeba8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:03.301743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440a05-6621-4317-83c5-e38bc4cbeba8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:03.301743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440a05-6621-4317-83c5-e38bc4cbeba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:03.375544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f5440a05-6621-4317-83c5-e38bc4cbeba8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:03.376845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f5440a05-6621-4317-83c5-e38bc4cbeba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:03.431131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:03.496315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 756/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:01:03.794584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1227b-5244-444f-b04e-6f4c9dfc0c40 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:03.796168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6eb1227b-5244-444f-b04e-6f4c9dfc0c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:03.821270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:03.910403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1227b-5244-444f-b04e-6f4c9dfc0c40 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:03.995606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368de11-135c-415a-bc63-8ac3516fe89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:03.996240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368de11-135c-415a-bc63-8ac3516fe89e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:03.996593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368de11-135c-415a-bc63-8ac3516fe89e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:03.996905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368de11-135c-415a-bc63-8ac3516fe89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:03.998643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5368de11-135c-415a-bc63-8ac3516fe89e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:04.087838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:04.131990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368de11-135c-415a-bc63-8ac3516fe89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:04.133577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368de11-135c-415a-bc63-8ac3516fe89e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:04.274507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 728/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:01:06.537702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941a00-9ed1-4691-a188-d43eb14ef902 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.538333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941a00-9ed1-4691-a188-d43eb14ef902 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.538778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941a00-9ed1-4691-a188-d43eb14ef902 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.541195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 757/1485] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 13:01:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:06.547383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08fff6-5c41-4c9e-afd6-b0f412ca9109 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.547928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08fff6-5c41-4c9e-afd6-b0f412ca9109 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.547928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08fff6-5c41-4c9e-afd6-b0f412ca9109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.621177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dc08fff6-5c41-4c9e-afd6-b0f412ca9109 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:06.622521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc08fff6-5c41-4c9e-afd6-b0f412ca9109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:06.729794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 729/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:01:07.021753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941a00-9ed1-4691-a188-d43eb14ef902 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:07.023322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ba941a00-9ed1-4691-a188-d43eb14ef902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:07.138112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941a00-9ed1-4691-a188-d43eb14ef902 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:07.215129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:07.217554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d10773-c3f7-4be8-91c9-1a2e5abf3025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:07.218405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d10773-c3f7-4be8-91c9-1a2e5abf3025 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:07.218942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d10773-c3f7-4be8-91c9-1a2e5abf3025 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:07.219283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d10773-c3f7-4be8-91c9-1a2e5abf3025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:07.221263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c9d10773-c3f7-4be8-91c9-1a2e5abf3025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:07.347237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d10773-c3f7-4be8-91c9-1a2e5abf3025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.348618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d10773-c3f7-4be8-91c9-1a2e5abf3025 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.369685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 758/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:01:09.880851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da800dad-a5f6-4d1a-b854-b90c9d3b06f5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.881488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da800dad-a5f6-4d1a-b854-b90c9d3b06f5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.881488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da800dad-a5f6-4d1a-b854-b90c9d3b06f5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.882862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 730/1488] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 13:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:09.889789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2aab-0be5-4531-8ca0-e1d5e71bc8a9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.890502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2aab-0be5-4531-8ca0-e1d5e71bc8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.890502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2aab-0be5-4531-8ca0-e1d5e71bc8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.972822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ca3d2aab-0be5-4531-8ca0-e1d5e71bc8a9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:09.975130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ca3d2aab-0be5-4531-8ca0-e1d5e71bc8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:10.038682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:10.088146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 759/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:01:10.409323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da800dad-a5f6-4d1a-b854-b90c9d3b06f5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.410990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-da800dad-a5f6-4d1a-b854-b90c9d3b06f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:10.449004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:10.529783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da800dad-a5f6-4d1a-b854-b90c9d3b06f5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:10.617538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf38d19-196f-449b-bed0-75bbd254f594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:10.618184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf38d19-196f-449b-bed0-75bbd254f594 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.618461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf38d19-196f-449b-bed0-75bbd254f594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.618731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf38d19-196f-449b-bed0-75bbd254f594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.620354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cbf38d19-196f-449b-bed0-75bbd254f594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:10.708720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:10.761012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf38d19-196f-449b-bed0-75bbd254f594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:10.763510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf38d19-196f-449b-bed0-75bbd254f594 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:10.784145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 731/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:01:13.359382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f045-a9d6-48d6-9e1e-ab9e49216f8d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:13.359949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f045-a9d6-48d6-9e1e-ab9e49216f8d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:13.359949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f045-a9d6-48d6-9e1e-ab9e49216f8d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:13.361160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 760/1491] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 13:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:13.367277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035bc83-aac2-4696-8625-f27a567075fe None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:13.368212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035bc83-aac2-4696-8625-f27a567075fe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:13.368212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035bc83-aac2-4696-8625-f27a567075fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:13.445690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2035bc83-aac2-4696-8625-f27a567075fe None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:13.446948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2035bc83-aac2-4696-8625-f27a567075fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:13.564296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 732/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:01:13.862258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f045-a9d6-48d6-9e1e-ab9e49216f8d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:13.864076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0614f045-a9d6-48d6-9e1e-ab9e49216f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:13.984477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f045-a9d6-48d6-9e1e-ab9e49216f8d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:14.065502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe9ae4-8994-4a83-9e10-ee2060c50e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:14.066264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe9ae4-8994-4a83-9e10-ee2060c50e0c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:14.066545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe9ae4-8994-4a83-9e10-ee2060c50e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:14.066828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe9ae4-8994-4a83-9e10-ee2060c50e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:14.068516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7cbe9ae4-8994-4a83-9e10-ee2060c50e0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:14.078727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:14.225846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe9ae4-8994-4a83-9e10-ee2060c50e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:14.228683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe9ae4-8994-4a83-9e10-ee2060c50e0c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:14.251265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 761/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:01:23.562274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=82 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:24.248640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=50 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:29.083600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:29.084233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:29.084233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:29.172632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:29.173904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:29.278738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 733/1494] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:02:29.286859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdae0-7314-4dd9-96c5-9af769a8db8f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:29.287579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdae0-7314-4dd9-96c5-9af769a8db8f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:29.287579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdae0-7314-4dd9-96c5-9af769a8db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:29.370314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-303bdae0-7314-4dd9-96c5-9af769a8db8f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:29.371513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-303bdae0-7314-4dd9-96c5-9af769a8db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:29.472145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 762/1495] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:02:29.477319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:29.478597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:29.490167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5a4f-db8e-4c76-a9e7-8f274e03249f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:29.557465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:29.594501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:29.595109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:29.595391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:29.595665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:29.596334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:29.596633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:29.597649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:29.597649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:02:29.597649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:02:29.598574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:29.610533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 734/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:29.619659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bdae0-7314-4dd9-96c5-9af769a8db8f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:29.629106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bdae0-7314-4dd9-96c5-9af769a8db8f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:29.694534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:29.729588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:29.730265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:29.730537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:29.730788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:29.731411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:29.733163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:29.733163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Sep 07 13:02:29.733163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: Sep 07 13:02:29.734387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:29.794984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 763/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:02:29 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:02:29.799504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:29.801006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:29.837750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d97877-7223-487c-b4e8-69e260e20bf8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:29.939107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:29.939718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:29.939991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:29.940300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:29.940878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:29.942018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:29.942018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:02:29.942018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:02:29.943124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:29.986508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 735/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:02:29 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:02:29.991132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:29.992847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:30.058788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7784-fd7e-476e-8e48-dd659a4f1cb4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:30.150307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:30.163123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:30.163748 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.164217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.164398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.165049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:30.166113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:30.166113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Sep 07 13:02:30.166113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: Sep 07 13:02:30.166996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:30.180183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 764/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:02:30.184451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:30.186103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:30.198450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdf535-057d-481b-83de-6e2499cc49b6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:30.226654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:30.305411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484de7b-4dba-4f0e-9b64-ada68bc2bf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:30.306051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484de7b-4dba-4f0e-9b64-ada68bc2bf3e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.306361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484de7b-4dba-4f0e-9b64-ada68bc2bf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.306648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484de7b-4dba-4f0e-9b64-ada68bc2bf3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.307276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484de7b-4dba-4f0e-9b64-ada68bc2bf3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:30.308290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-4484de7b-4dba-4f0e-9b64-ada68bc2bf3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:30.308290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:02:30.308290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:02:30.309209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484de7b-4dba-4f0e-9b64-ada68bc2bf3e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:30.322340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 736/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:02:30.327067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:30.328638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:30.365265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dea3cf-53a9-4d04-a0a5-d121e8af6958 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:30.466413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e559129-3349-45fe-9a3b-cab119e0a0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:30.467112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559129-3349-45fe-9a3b-cab119e0a0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.467365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559129-3349-45fe-9a3b-cab119e0a0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.467806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559129-3349-45fe-9a3b-cab119e0a0c4 admin admin] PATH_INFO: `/v3/projects/3442bc1b48874b8292c9983e7d9e0ac5/users/70d5cf6e3e5f4e088cb8ec2f53695d28/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.492222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e559129-3349-45fe-9a3b-cab119e0a0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3442bc1b48874b8292c9983e7d9e0ac5, user_id=70d5cf6e3e5f4e088cb8ec2f53695d28, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:30.493275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-9e559129-3349-45fe-9a3b-cab119e0a0c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:30.493275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Sep 07 13:02:30.493275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: Sep 07 13:02:30.495246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e559129-3349-45fe-9a3b-cab119e0a0c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:30.522210 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 765/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:02:30 2022] PUT /identity/v3/projects/3442bc1b48874b8292c9983e7d9e0ac5/users/70d5cf6e3e5f4e088cb8ec2f53695d28/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:02:30.535122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475088a-5f60-4e64-934e-f64d16dc4287 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.535430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475088a-5f60-4e64-934e-f64d16dc4287 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.535748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475088a-5f60-4e64-934e-f64d16dc4287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.596985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2475088a-5f60-4e64-934e-f64d16dc4287 None None] MFA Rules not processed for user `70d5cf6e3e5f4e088cb8ec2f53695d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:30.598602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2475088a-5f60-4e64-934e-f64d16dc4287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:30.681838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 737/1502] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:02:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:02:30.695822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451eb2ba-01a6-419c-bfa5-71206d51b0ec None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.696157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451eb2ba-01a6-419c-bfa5-71206d51b0ec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.696426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451eb2ba-01a6-419c-bfa5-71206d51b0ec None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.697879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 766/1503] 10.222.0.17 () {54 vars in 964 bytes} [Wed Sep 7 13:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:30.703237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4322-a1fa-4ec7-b510-bace126ca2e4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.703529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4322-a1fa-4ec7-b510-bace126ca2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.703805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4322-a1fa-4ec7-b510-bace126ca2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.738723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:30.791091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c43a4322-a1fa-4ec7-b510-bace126ca2e4 None None] MFA Rules not processed for user `42f946fc56244e179973d6f846eb7548`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:30.792368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c43a4322-a1fa-4ec7-b510-bace126ca2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:30.876021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 738/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Sep 7 13:02:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:02:30.884377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451eb2ba-01a6-419c-bfa5-71206d51b0ec None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:30.885937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-451eb2ba-01a6-419c-bfa5-71206d51b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:30.946973 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:31.005021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451eb2ba-01a6-419c-bfa5-71206d51b0ec None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:31.083697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:31.084369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:31.084673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:31.084939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:31.086585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:31.190776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:31.192252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:31.195752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:31.210800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 767/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:02:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:02:31.257764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4361-32dc-4cc9-8bed-7f124253d1a6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:31.258237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4361-32dc-4cc9-8bed-7f124253d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:31.258237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4361-32dc-4cc9-8bed-7f124253d1a6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:31.259517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 739/1506] 10.222.0.17 () {52 vars in 830 bytes} [Wed Sep 7 13:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:31.263885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:31.265119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:31.371276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6221-8b8f-4e09-8f41-db7dd48add1c service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:31.484602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:31.485242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:31.485518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:31.485825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] PATH_INFO: `/v3/users/70d5cf6e3e5f4e088cb8ec2f53695d28` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:31.500401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d5cf6e3e5f4e088cb8ec2f53695d28)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:31.501449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:31.501449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Sep 07 13:02:31.501449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: Sep 07 13:02:31.502712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:31.513930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:31.576465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d5cf6e3e5f4e088cb8ec2f53695d28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:31.599841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d5cf6e3e5f4e088cb8ec2f53695d28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:31.610647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d5cf6e3e5f4e088cb8ec2f53695d28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:31.611170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d5cf6e3e5f4e088cb8ec2f53695d28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:31.626568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d5cf6e3e5f4e088cb8ec2f53695d28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:31.632914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 768/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:02:31 2022] DELETE /identity/v3/users/70d5cf6e3e5f4e088cb8ec2f53695d28 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:02:31.637501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:31.637813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:31.638085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:31.697681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:31.698919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:31.786753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:31.802504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 740/1508] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:02:31.814701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:31.816557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:31.929753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d250c-8b90-4e2d-ae72-657f0f72e59b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:31.988976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:32.012874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536348b-d366-46f8-b0d8-dc06b65bde9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:32.014058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536348b-d366-46f8-b0d8-dc06b65bde9d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:32.014467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536348b-d366-46f8-b0d8-dc06b65bde9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:32.014854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536348b-d366-46f8-b0d8-dc06b65bde9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:32.016948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6536348b-d366-46f8-b0d8-dc06b65bde9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:32.148078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536348b-d366-46f8-b0d8-dc06b65bde9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:32.150842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536348b-d366-46f8-b0d8-dc06b65bde9d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:32.172197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 769/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:02:32.515039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:32.516530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:32.529435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031924f0-43c8-49a9-becd-c3fe1d52dd3a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:32.634665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:32.635314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:32.635643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:32.635884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] PATH_INFO: `/v3/projects/3442bc1b48874b8292c9983e7d9e0ac5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:32.647587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3442bc1b48874b8292c9983e7d9e0ac5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:32.648636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:02:32.648636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:02:32.648636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:02:32.649874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:32.713372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3442bc1b48874b8292c9983e7d9e0ac5', 'request_id': 'req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:32.719854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3442bc1b48874b8292c9983e7d9e0ac5', 'request_id': 'req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:32.733615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3442bc1b48874b8292c9983e7d9e0ac5', 'request_id': 'req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:32.766752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] The token cache is being invalidate because project 3442bc1b48874b8292c9983e7d9e0ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:02:32.767276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:32.767737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-934b0a84-9c0c-48e7-922b-465e5dd6b7c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:02:32.769157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 741/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:02:32 2022] DELETE /identity/v3/projects/3442bc1b48874b8292c9983e7d9e0ac5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:02:42.169962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=62 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:42.764271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:09.814989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f008cd-c060-4d67-b569-ba270b19970b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:09.815738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f008cd-c060-4d67-b569-ba270b19970b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:09.815738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f008cd-c060-4d67-b569-ba270b19970b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:09.817002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 770/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:09.826342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5f108-1d89-4fc4-86db-2b96d4578304 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:09.826929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5f108-1d89-4fc4-86db-2b96d4578304 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:09.826929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5f108-1d89-4fc4-86db-2b96d4578304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:09.923236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-41f5f108-1d89-4fc4-86db-2b96d4578304 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:09.924496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-41f5f108-1d89-4fc4-86db-2b96d4578304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:10.040061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 742/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:10.202007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f008cd-c060-4d67-b569-ba270b19970b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:10.204432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-06f008cd-c060-4d67-b569-ba270b19970b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:10.334177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f008cd-c060-4d67-b569-ba270b19970b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:10.340285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed09abd-bb72-4278-93e5-e7e4e8ebf9d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:10.340930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed09abd-bb72-4278-93e5-e7e4e8ebf9d0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:10.341210 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed09abd-bb72-4278-93e5-e7e4e8ebf9d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:10.341486 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed09abd-bb72-4278-93e5-e7e4e8ebf9d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:10.350389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed09abd-bb72-4278-93e5-e7e4e8ebf9d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:10.352002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed09abd-bb72-4278-93e5-e7e4e8ebf9d0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:10.361650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 771/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:11.447966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe1a7-b1e9-4454-a5eb-f3d78826ce8f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:11.448568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe1a7-b1e9-4454-a5eb-f3d78826ce8f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:11.448568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe1a7-b1e9-4454-a5eb-f3d78826ce8f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:11.449866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 743/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:11.456638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625463d-b1e9-4e3f-8785-2adb5b0fdf73 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:11.457131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625463d-b1e9-4e3f-8785-2adb5b0fdf73 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:11.457131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625463d-b1e9-4e3f-8785-2adb5b0fdf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:11.478202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3625463d-b1e9-4e3f-8785-2adb5b0fdf73 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:11.479745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3625463d-b1e9-4e3f-8785-2adb5b0fdf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:11.489000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 772/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:11.642539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe1a7-b1e9-4454-a5eb-f3d78826ce8f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:11.658580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe1a7-b1e9-4454-a5eb-f3d78826ce8f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:11.665168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bc9a6-4c13-486f-aa7a-4a99784390d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:11.665808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bc9a6-4c13-486f-aa7a-4a99784390d6 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:11.666208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bc9a6-4c13-486f-aa7a-4a99784390d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:11.666338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bc9a6-4c13-486f-aa7a-4a99784390d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:11.675439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bc9a6-4c13-486f-aa7a-4a99784390d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:11.677030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bc9a6-4c13-486f-aa7a-4a99784390d6 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:11.686937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 744/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:13.036342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd164-7d08-42d7-ba41-46185d68f165 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:13.037051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd164-7d08-42d7-ba41-46185d68f165 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:13.037051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd164-7d08-42d7-ba41-46185d68f165 None None] PATH_INFO: `/v3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:13.038269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 773/1517] 10.222.0.59 () {54 vars in 925 bytes} [Wed Sep 7 13:04:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:13.046246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcb775-0a41-4d87-9e36-93835c1d64d2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:13.046754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcb775-0a41-4d87-9e36-93835c1d64d2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:13.046754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcb775-0a41-4d87-9e36-93835c1d64d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:13.116911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-40bcb775-0a41-4d87-9e36-93835c1d64d2 None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:13.118210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-40bcb775-0a41-4d87-9e36-93835c1d64d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:13.137985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:13.201007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 745/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Sep 7 13:04:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:21.472714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=56 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:23.197940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=23 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:23.227827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686c9f-9643-4ae4-8a7e-719b2f21faa7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.228488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686c9f-9643-4ae4-8a7e-719b2f21faa7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.228582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686c9f-9643-4ae4-8a7e-719b2f21faa7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.230727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 774/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:23.240185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb35bbe-9919-4990-aa02-dc4fa773d25c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.241219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb35bbe-9919-4990-aa02-dc4fa773d25c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.241219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb35bbe-9919-4990-aa02-dc4fa773d25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.269715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8fb35bbe-9919-4990-aa02-dc4fa773d25c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:23.271020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8fb35bbe-9919-4990-aa02-dc4fa773d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:23.281121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 746/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:23.323851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca974-72bf-4584-9169-ba1e4e4a757e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.324137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca974-72bf-4584-9169-ba1e4e4a757e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.324352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca974-72bf-4584-9169-ba1e4e4a757e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.325745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 775/1521] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 13:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:23.331490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f741f4-5862-48aa-b297-2530f5f678aa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.331780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f741f4-5862-48aa-b297-2530f5f678aa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.332058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f741f4-5862-48aa-b297-2530f5f678aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.355200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-32f741f4-5862-48aa-b297-2530f5f678aa None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:23.356490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-32f741f4-5862-48aa-b297-2530f5f678aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:23.366678 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 747/1522] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 13:04:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:23.374819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddca974-72bf-4584-9169-ba1e4e4a757e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:23.394521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddca974-72bf-4584-9169-ba1e4e4a757e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:23.404589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64025c3-71a2-42ae-a9ea-aac2b7a093d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:23.405240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64025c3-71a2-42ae-a9ea-aac2b7a093d5 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.405470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64025c3-71a2-42ae-a9ea-aac2b7a093d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.405795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64025c3-71a2-42ae-a9ea-aac2b7a093d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.415465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64025c3-71a2-42ae-a9ea-aac2b7a093d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:23.417073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64025c3-71a2-42ae-a9ea-aac2b7a093d5 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:23.428584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 776/1523] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:23.452611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f741f4-5862-48aa-b297-2530f5f678aa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:23.466555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f741f4-5862-48aa-b297-2530f5f678aa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:23.472870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd10d4-101b-42b4-98a9-b265b2eaf8ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:23.473501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd10d4-101b-42b4-98a9-b265b2eaf8ac service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.473963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd10d4-101b-42b4-98a9-b265b2eaf8ac service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.474239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd10d4-101b-42b4-98a9-b265b2eaf8ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.483660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd10d4-101b-42b4-98a9-b265b2eaf8ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:23.485522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd10d4-101b-42b4-98a9-b265b2eaf8ac service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:23.496655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 748/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:24.612490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9af246-77c4-488c-b3e1-1b174a5d320e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:24.613099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9af246-77c4-488c-b3e1-1b174a5d320e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:24.613207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9af246-77c4-488c-b3e1-1b174a5d320e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:24.614723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 777/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:24.621572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582824e-8223-4745-97a1-781637de7e59 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:24.622111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582824e-8223-4745-97a1-781637de7e59 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:24.622218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582824e-8223-4745-97a1-781637de7e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:24.643375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3582824e-8223-4745-97a1-781637de7e59 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:24.644515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3582824e-8223-4745-97a1-781637de7e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:24.652848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 749/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:24.801918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9af246-77c4-488c-b3e1-1b174a5d320e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:24.818138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9af246-77c4-488c-b3e1-1b174a5d320e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:24.824336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac577df-84b0-4baf-861d-fbc86d86f480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:24.825023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac577df-84b0-4baf-861d-fbc86d86f480 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:24.825250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac577df-84b0-4baf-861d-fbc86d86f480 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:24.825517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac577df-84b0-4baf-861d-fbc86d86f480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:24.835399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac577df-84b0-4baf-861d-fbc86d86f480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:24.837015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac577df-84b0-4baf-861d-fbc86d86f480 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:24.848477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 778/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:29.052743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3641f16-78a1-43cb-86ca-b3185b73452f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:29.053348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3641f16-78a1-43cb-86ca-b3185b73452f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:29.053348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3641f16-78a1-43cb-86ca-b3185b73452f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:29.054688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 750/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:29.060535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4031c-3f65-46b9-9dba-a452c28fbb15 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:29.061073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4031c-3f65-46b9-9dba-a452c28fbb15 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:29.061073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4031c-3f65-46b9-9dba-a452c28fbb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:29.081591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4fb4031c-3f65-46b9-9dba-a452c28fbb15 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:29.082833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4fb4031c-3f65-46b9-9dba-a452c28fbb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:29.091064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 779/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:29.374816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3641f16-78a1-43cb-86ca-b3185b73452f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:29.376674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d3641f16-78a1-43cb-86ca-b3185b73452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:29.462771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3641f16-78a1-43cb-86ca-b3185b73452f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:29.466681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3e313-0315-416d-aec4-5799aec29727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:29.467299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3e313-0315-416d-aec4-5799aec29727 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:29.467570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3e313-0315-416d-aec4-5799aec29727 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:29.467842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3e313-0315-416d-aec4-5799aec29727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:29.477897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3e313-0315-416d-aec4-5799aec29727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:29.479339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3e313-0315-416d-aec4-5799aec29727 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:29.490039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 751/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:31.660059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b26a11-9fa2-48f5-a96f-a558e75fa3ae None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:31.660598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b26a11-9fa2-48f5-a96f-a558e75fa3ae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:31.660598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b26a11-9fa2-48f5-a96f-a558e75fa3ae None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:31.661923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 780/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:31.668151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916ff7-5697-47c0-bf24-c4f82e9a3f47 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:31.668622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916ff7-5697-47c0-bf24-c4f82e9a3f47 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:31.668717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916ff7-5697-47c0-bf24-c4f82e9a3f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:31.690780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e9916ff7-5697-47c0-bf24-c4f82e9a3f47 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:31.691984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e9916ff7-5697-47c0-bf24-c4f82e9a3f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:31.700015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 752/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:31.988596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b26a11-9fa2-48f5-a96f-a558e75fa3ae None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:32.006585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b26a11-9fa2-48f5-a96f-a558e75fa3ae None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:32.010695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790b885-9727-4519-a6ac-5018fbce12fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:32.011327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790b885-9727-4519-a6ac-5018fbce12fc service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:32.011599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790b885-9727-4519-a6ac-5018fbce12fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:32.011866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790b885-9727-4519-a6ac-5018fbce12fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:32.022664 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790b885-9727-4519-a6ac-5018fbce12fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:32.024146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790b885-9727-4519-a6ac-5018fbce12fc service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:32.036794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 781/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:35.812061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef62926-3395-4364-8e39-9252df1c9ce6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:35.812673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef62926-3395-4364-8e39-9252df1c9ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:35.812673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef62926-3395-4364-8e39-9252df1c9ce6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:35.814152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 753/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:35.820571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110318d-1043-4dc4-b465-67d815370dc2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:35.821047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110318d-1043-4dc4-b465-67d815370dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:35.821162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110318d-1043-4dc4-b465-67d815370dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:35.844180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f110318d-1043-4dc4-b465-67d815370dc2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:35.845383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f110318d-1043-4dc4-b465-67d815370dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:35.853946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 782/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:36.158118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef62926-3395-4364-8e39-9252df1c9ce6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:36.160760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aef62926-3395-4364-8e39-9252df1c9ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:36.252575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef62926-3395-4364-8e39-9252df1c9ce6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:36.256384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afa25e-7f0b-4daf-b207-304b6db81c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:36.256987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afa25e-7f0b-4daf-b207-304b6db81c51 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:36.257263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afa25e-7f0b-4daf-b207-304b6db81c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:36.257534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afa25e-7f0b-4daf-b207-304b6db81c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:36.267395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afa25e-7f0b-4daf-b207-304b6db81c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:36.268968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afa25e-7f0b-4daf-b207-304b6db81c51 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:36.278814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 754/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:38.496329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e98ac-d3ca-46dc-b622-0bf936f8cd08 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:38.497197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e98ac-d3ca-46dc-b622-0bf936f8cd08 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:38.497197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e98ac-d3ca-46dc-b622-0bf936f8cd08 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:38.499352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 783/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:04:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:38.508611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eefa4-ba76-4e9c-8f0e-ea6d07ccabc7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:38.509170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eefa4-ba76-4e9c-8f0e-ea6d07ccabc7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:38.509170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eefa4-ba76-4e9c-8f0e-ea6d07ccabc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:38.539900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b94eefa4-ba76-4e9c-8f0e-ea6d07ccabc7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:38.541947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b94eefa4-ba76-4e9c-8f0e-ea6d07ccabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:38.555283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 755/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:04:38.897985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e98ac-d3ca-46dc-b622-0bf936f8cd08 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:38.900635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2c7e98ac-d3ca-46dc-b622-0bf936f8cd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:38.919056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e98ac-d3ca-46dc-b622-0bf936f8cd08 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:38.923269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0ddc2-ebb0-4acc-acb5-5d9514f9701a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:38.923871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0ddc2-ebb0-4acc-acb5-5d9514f9701a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:38.924177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0ddc2-ebb0-4acc-acb5-5d9514f9701a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:38.924467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0ddc2-ebb0-4acc-acb5-5d9514f9701a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:38.934090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0ddc2-ebb0-4acc-acb5-5d9514f9701a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:38.935537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0ddc2-ebb0-4acc-acb5-5d9514f9701a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:38.945977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 784/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:04:48.532826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:48.941327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:06.701881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fdf72-abfb-4d2e-85da-7bf737808e70 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:06.702539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fdf72-abfb-4d2e-85da-7bf737808e70 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:06.702539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fdf72-abfb-4d2e-85da-7bf737808e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:06.726655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2a4fdf72-abfb-4d2e-85da-7bf737808e70 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:06.727855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2a4fdf72-abfb-4d2e-85da-7bf737808e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:06.740927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 756/1540] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:06.749918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e7153-7da8-4760-9bcb-ca2ee5090a3d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:06.751604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e7153-7da8-4760-9bcb-ca2ee5090a3d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:06.751604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e7153-7da8-4760-9bcb-ca2ee5090a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:06.786884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-736e7153-7da8-4760-9bcb-ca2ee5090a3d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:06.788163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-736e7153-7da8-4760-9bcb-ca2ee5090a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:06.801777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 785/1541] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:06.805992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fdf72-abfb-4d2e-85da-7bf737808e70 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:06.824255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fdf72-abfb-4d2e-85da-7bf737808e70 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:06.834429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:06.835060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:06.835338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:06.836183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:06.836295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:06.836527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:06.838174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:06.861617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 757/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:06.869595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e7153-7da8-4760-9bcb-ca2ee5090a3d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:06.881176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e7153-7da8-4760-9bcb-ca2ee5090a3d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:06.891358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd6b41-a179-4341-871c-d6d64ffce12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:06.892240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd6b41-a179-4341-871c-d6d64ffce12f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:06.892717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd6b41-a179-4341-871c-d6d64ffce12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:06.893836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd6b41-a179-4341-871c-d6d64ffce12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:06.894662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd6b41-a179-4341-871c-d6d64ffce12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:06.896897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd6b41-a179-4341-871c-d6d64ffce12f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:06.952586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 786/1543] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:06 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:06.956662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:06.975424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3f555-5627-44e0-bfaa-193e6f2a1622 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.068639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.069278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.069558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.069860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.070506 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.072460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.083958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9b77-de59-4c06-8d40-c3ebf58244e1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.084594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9b77-de59-4c06-8d40-c3ebf58244e1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.085095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9b77-de59-4c06-8d40-c3ebf58244e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.104394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 758/1544] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:06 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:07.108403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.123465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f123-e768-4197-9b95-8142b7bc3e85 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.126683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-55ef9b77-de59-4c06-8d40-c3ebf58244e1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.129078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-55ef9b77-de59-4c06-8d40-c3ebf58244e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:07.138974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.140345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.140828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.141354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.142543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.149637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 787/1545] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:07.149867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.160621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a796622-3997-4f15-b945-029f32d59d81 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.161285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a796622-3997-4f15-b945-029f32d59d81 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.161830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a796622-3997-4f15-b945-029f32d59d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.169775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 759/1546] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:06:07.173367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.196682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5a796622-3997-4f15-b945-029f32d59d81 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.198219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a796622-3997-4f15-b945-029f32d59d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:07.200959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db4080-47ec-4dad-b97b-5a3719946aed admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.214102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 788/1547] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:07.214730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.215547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.215769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930513-51a1-4191-850c-9b0962b615b5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.216155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.216387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930513-51a1-4191-850c-9b0962b615b5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.216763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.216894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930513-51a1-4191-850c-9b0962b615b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.217915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.219687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.241992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 760/1548] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:06:07.242894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.260866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fd930513-51a1-4191-850c-9b0962b615b5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.262101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fd930513-51a1-4191-850c-9b0962b615b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:07.275043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f1a07-ec77-457c-bcdf-ca3e94375f78 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.284653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 789/1549] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:07.286641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930513-51a1-4191-850c-9b0962b615b5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.290452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbd65e-aaae-432e-8fe5-f1c208fc3e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.291534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbd65e-aaae-432e-8fe5-f1c208fc3e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.291900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbd65e-aaae-432e-8fe5-f1c208fc3e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.292181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbd65e-aaae-432e-8fe5-f1c208fc3e12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.292843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbd65e-aaae-432e-8fe5-f1c208fc3e12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:07.293150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbd65e-aaae-432e-8fe5-f1c208fc3e12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.294813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbd65e-aaae-432e-8fe5-f1c208fc3e12 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.304633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930513-51a1-4191-850c-9b0962b615b5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.313535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.314256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.314818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.315114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] PATH_INFO: `/v3/projects/9a847ae68ca242c0bce510fd91c2633c/users/c0db39216fd24b5f9cb0531bcedb0326/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.319380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 761/1550] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:07.320864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260b060-0dad-4dc8-b488-64f2a791591a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.321417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260b060-0dad-4dc8-b488-64f2a791591a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.322016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260b060-0dad-4dc8-b488-64f2a791591a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.341775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a847ae68ca242c0bce510fd91c2633c, user_id=c0db39216fd24b5f9cb0531bcedb0326, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.346015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.361844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1260b060-0dad-4dc8-b488-64f2a791591a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.363168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1260b060-0dad-4dc8-b488-64f2a791591a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:07.371064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 790/1551] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:07 2022] PUT /identity/v3/projects/9a847ae68ca242c0bce510fd91c2633c/users/c0db39216fd24b5f9cb0531bcedb0326/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:07.372494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.492071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6abcb-5b2a-4968-9f7a-325b8b7f9749 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.515068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4be8e-5797-427c-b9e4-db6d6a728664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.515716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4be8e-5797-427c-b9e4-db6d6a728664 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.515996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4be8e-5797-427c-b9e4-db6d6a728664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.516266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4be8e-5797-427c-b9e4-db6d6a728664 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.516912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4be8e-5797-427c-b9e4-db6d6a728664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.519568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4be8e-5797-427c-b9e4-db6d6a728664 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.534653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 762/1552] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:07.536107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f392fa-8c6b-451a-8e2c-9311304eec84 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.536393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f392fa-8c6b-451a-8e2c-9311304eec84 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.536668 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f392fa-8c6b-451a-8e2c-9311304eec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.583989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-22f392fa-8c6b-451a-8e2c-9311304eec84 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.584698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 791/1553] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/projects => generated 391 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:07.586070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7012d8-3000-4766-aba8-977f3bd20bae None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.586357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7012d8-3000-4766-aba8-977f3bd20bae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.586632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7012d8-3000-4766-aba8-977f3bd20bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.588030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-22f392fa-8c6b-451a-8e2c-9311304eec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:07.626882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3a7012d8-3000-4766-aba8-977f3bd20bae None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.628709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3a7012d8-3000-4766-aba8-977f3bd20bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:07.667632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:07.746088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 763/1554] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:07.746890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f392fa-8c6b-451a-8e2c-9311304eec84 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.759629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f392fa-8c6b-451a-8e2c-9311304eec84 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.763758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 792/1555] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:07.764973 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7012d8-3000-4766-aba8-977f3bd20bae None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.779122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dad268-5d73-4da3-b24a-e79ef592e181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.779831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dad268-5d73-4da3-b24a-e79ef592e181 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.780096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dad268-5d73-4da3-b24a-e79ef592e181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.780369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dad268-5d73-4da3-b24a-e79ef592e181 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.781025 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dad268-5d73-4da3-b24a-e79ef592e181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:07.781401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dad268-5d73-4da3-b24a-e79ef592e181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.783419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dad268-5d73-4da3-b24a-e79ef592e181 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.785011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:07.793807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7012d8-3000-4766-aba8-977f3bd20bae None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.798871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 764/1556] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:07.800169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd71b1e-b60a-4ac5-915d-13fa4510ac97 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.800453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd71b1e-b60a-4ac5-915d-13fa4510ac97 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.800726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd71b1e-b60a-4ac5-915d-13fa4510ac97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.811163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.811799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.812081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.812353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.812991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.814437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:06:07.814437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Sep 07 13:06:07.814437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: Sep 07 13:06:07.815337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.844172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4bd71b1e-b60a-4ac5-915d-13fa4510ac97 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.845495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4bd71b1e-b60a-4ac5-915d-13fa4510ac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:07.869635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 793/1557] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:07.870228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 765/1558] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:07.870754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.871151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd71b1e-b60a-4ac5-915d-13fa4510ac97 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:07.886919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd71b1e-b60a-4ac5-915d-13fa4510ac97 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.899220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd063e-ad45-465e-8a2e-488919dca9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:07.900152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd063e-ad45-465e-8a2e-488919dca9ec admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.902425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd063e-ad45-465e-8a2e-488919dca9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.902705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd063e-ad45-465e-8a2e-488919dca9ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.903357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd063e-ad45-465e-8a2e-488919dca9ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:07.905234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-5cbd063e-ad45-465e-8a2e-488919dca9ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:06:07.905234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:06:07.905234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:06:07.906460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd063e-ad45-465e-8a2e-488919dca9ec admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:07.950459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 766/1559] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/projects => generated 405 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:07.952427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b337de-3cb8-44f7-bbf3-430e061df641 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:07.952721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b337de-3cb8-44f7-bbf3-430e061df641 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:07.952999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b337de-3cb8-44f7-bbf3-430e061df641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:07.960955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edda2b4-745b-4a60-9b8f-8afceab3e130 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:07.976494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a5b337de-3cb8-44f7-bbf3-430e061df641 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:07.977793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a5b337de-3cb8-44f7-bbf3-430e061df641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.013353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a4184-2154-4bbf-a94c-e58573a285db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.013919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4184-2154-4bbf-a94c-e58573a285db service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.014185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4184-2154-4bbf-a94c-e58573a285db service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.014753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4184-2154-4bbf-a94c-e58573a285db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.041347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 767/1560] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.042222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b337de-3cb8-44f7-bbf3-430e061df641 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.053888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b337de-3cb8-44f7-bbf3-430e061df641 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.058905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.059521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.059801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.060073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.060738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.062384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.073826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 768/1561] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:08.077719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.085516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a4184-2154-4bbf-a94c-e58573a285db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.087204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a4184-2154-4bbf-a94c-e58573a285db service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.087959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050040e-b001-4142-aab1-06f8eeafd4c6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.091684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.092276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.092552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.092820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.093450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:08.093809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.095402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.096991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 794/1562] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:08.098511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb99b0-04d7-4a94-b831-7bc965ebcef3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.098870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb99b0-04d7-4a94-b831-7bc965ebcef3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.099076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb99b0-04d7-4a94-b831-7bc965ebcef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.118749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 769/1563] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:08.118749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.126990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-55eb99b0-04d7-4a94-b831-7bc965ebcef3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.128490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-55eb99b0-04d7-4a94-b831-7bc965ebcef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.132346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ed5f4-03aa-431d-9405-8ca8b3f8f770 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.145177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 795/1564] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.145411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2610c-0b24-49d5-8f8b-329bed8e0e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.145411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2610c-0b24-49d5-8f8b-329bed8e0e50 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.145629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e347e5-7046-437e-8c25-464f40ee87b5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.145748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2610c-0b24-49d5-8f8b-329bed8e0e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.145748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2610c-0b24-49d5-8f8b-329bed8e0e50 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.145937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e347e5-7046-437e-8c25-464f40ee87b5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.145937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e347e5-7046-437e-8c25-464f40ee87b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.146108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2610c-0b24-49d5-8f8b-329bed8e0e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.146108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2610c-0b24-49d5-8f8b-329bed8e0e50 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.166068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d7e347e5-7046-437e-8c25-464f40ee87b5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.167417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7e347e5-7046-437e-8c25-464f40ee87b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.169655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 770/1565] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/users => generated 303 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:08.174415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac02800-47a0-4d93-81f5-819d0a635912 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.174792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac02800-47a0-4d93-81f5-819d0a635912 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.175127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac02800-47a0-4d93-81f5-819d0a635912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.177826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 796/1566] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.179163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8775b-b8b3-4eeb-adf4-39eb7bf3080a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.179667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8775b-b8b3-4eeb-adf4-39eb7bf3080a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.179788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8775b-b8b3-4eeb-adf4-39eb7bf3080a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.206634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c5b8775b-b8b3-4eeb-adf4-39eb7bf3080a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.208025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c5b8775b-b8b3-4eeb-adf4-39eb7bf3080a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.212765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9ac02800-47a0-4d93-81f5-819d0a635912 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.214112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9ac02800-47a0-4d93-81f5-819d0a635912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.216874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 797/1567] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.218028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8775b-b8b3-4eeb-adf4-39eb7bf3080a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.225743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 771/1568] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.225743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac02800-47a0-4d93-81f5-819d0a635912 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.230866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8775b-b8b3-4eeb-adf4-39eb7bf3080a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.236384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399732a-9a68-455b-8679-a267566fcf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.237411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399732a-9a68-455b-8679-a267566fcf02 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.237411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399732a-9a68-455b-8679-a267566fcf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.237663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399732a-9a68-455b-8679-a267566fcf02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.238346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399732a-9a68-455b-8679-a267566fcf02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.238755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac02800-47a0-4d93-81f5-819d0a635912 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.239965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399732a-9a68-455b-8679-a267566fcf02 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.244939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edfc38-3169-4cb4-adf9-40c1b144060a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.245745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edfc38-3169-4cb4-adf9-40c1b144060a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.246177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edfc38-3169-4cb4-adf9-40c1b144060a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.246657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edfc38-3169-4cb4-adf9-40c1b144060a admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.247402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edfc38-3169-4cb4-adf9-40c1b144060a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.250176 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edfc38-3169-4cb4-adf9-40c1b144060a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.256883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 798/1569] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:08.258488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d8f83-654c-4dca-8ad9-0705eda7843b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.258891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d8f83-654c-4dca-8ad9-0705eda7843b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.259290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d8f83-654c-4dca-8ad9-0705eda7843b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.284596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6d8d8f83-654c-4dca-8ad9-0705eda7843b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.285964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6d8d8f83-654c-4dca-8ad9-0705eda7843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.296514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 799/1570] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.297020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 772/1571] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:08.298769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d8f83-654c-4dca-8ad9-0705eda7843b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.300763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20dba1-b49d-4c44-86c2-d9e4c16092f1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.301158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20dba1-b49d-4c44-86c2-d9e4c16092f1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.301788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20dba1-b49d-4c44-86c2-d9e4c16092f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.312542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d8f83-654c-4dca-8ad9-0705eda7843b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.326256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2b20dba1-b49d-4c44-86c2-d9e4c16092f1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.327494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b20dba1-b49d-4c44-86c2-d9e4c16092f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.328501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:08.352050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:08.386443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.387309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.387594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.387913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.388732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.390925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.408683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 800/1572] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:08.408683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.411461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 773/1573] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.417160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fdcaa-d49e-4791-821f-7de821c95119 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.417550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fdcaa-d49e-4791-821f-7de821c95119 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.418219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fdcaa-d49e-4791-821f-7de821c95119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.421546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adb419-c5d6-47f0-86e6-5b38e2352c52 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.430044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.431137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.431709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.432129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.433346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:08.433762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.435155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:06:08.435155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Sep 07 13:06:08.435155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: Sep 07 13:06:08.436393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.445049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b28fdcaa-d49e-4791-821f-7de821c95119 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.446824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b28fdcaa-d49e-4791-821f-7de821c95119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.451854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 801/1574] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:08.452795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.457387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 774/1575] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.458458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fdcaa-d49e-4791-821f-7de821c95119 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.469162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bf06e-317a-4336-86b1-90c495db3d49 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.472559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fdcaa-d49e-4791-821f-7de821c95119 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.475307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.475995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.476372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.477018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] PATH_INFO: `/v3/projects/9bb55accaa9046d99253a42a3154fe1b/users/3cad6d025da54eb29d09fd2a2888c844/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.478250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.478974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.479529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.479894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.481475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.483859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.500450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb55accaa9046d99253a42a3154fe1b, user_id=3cad6d025da54eb29d09fd2a2888c844, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.504337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.514895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 775/1576] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/users => generated 292 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:08.515703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.528513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0d097-b995-427f-8f35-1593260979e7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.530595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 802/1577] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:08 2022] PUT /identity/v3/projects/9bb55accaa9046d99253a42a3154fe1b/users/3cad6d025da54eb29d09fd2a2888c844/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:08.531732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.630832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.632085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.632788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.633691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.635361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:08.635977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.639237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd13d6-a190-4e7f-adba-a3bb6835f78d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.640232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.646182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.646817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.647284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.647546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.648576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.650472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.667134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 803/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:08.667429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 776/1579] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:08.667594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.667755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.676900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df16e8-3185-4167-b468-7d1171a6ff53 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.680655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bb189-fd7d-46b2-93f4-04779e2b7248 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.684014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.684690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.684986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.685399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.686086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.686944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.687564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.687844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.688050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.688164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.688933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:08.689242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.691667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.706569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 804/1580] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:08.713711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.731413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f1320-d965-459e-ba77-3f04ab1acc84 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.731723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 777/1581] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:08.731723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.803682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:08.812434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb23f3-ecc6-4340-8a94-b930da2fc08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.816102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb23f3-ecc6-4340-8a94-b930da2fc08d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.817179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb23f3-ecc6-4340-8a94-b930da2fc08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.818182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb23f3-ecc6-4340-8a94-b930da2fc08d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.819526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb23f3-ecc6-4340-8a94-b930da2fc08d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:08.820053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb23f3-ecc6-4340-8a94-b930da2fc08d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.822232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb23f3-ecc6-4340-8a94-b930da2fc08d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.833882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7273b2e-23eb-4919-b18a-b42501d27dd1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.845737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 805/1582] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:08.855122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbb976-b6c2-490b-a7f0-80ca9d9d80a2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.855491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbb976-b6c2-490b-a7f0-80ca9d9d80a2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.855693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbb976-b6c2-490b-a7f0-80ca9d9d80a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.857742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.858428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.858859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.859192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.860521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.861756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.878095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 778/1583] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:08.878095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.885634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12825d78-45a1-48c0-b6cc-dc04656cdc52 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.891481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.892251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.892714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.893228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] PATH_INFO: `/v3/projects/67b7595d39aa4603a78c2f6a2ab7408f/users/e508d401e9284e548470a54b05d91ec7/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.894935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-46fbb976-b6c2-490b-a7f0-80ca9d9d80a2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:08.896744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:08.897089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-46fbb976-b6c2-490b-a7f0-80ca9d9d80a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:08.912854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 806/1584] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:08.914093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbb976-b6c2-490b-a7f0-80ca9d9d80a2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.918931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b7595d39aa4603a78c2f6a2ab7408f, user_id=e508d401e9284e548470a54b05d91ec7, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.921024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:06:08.921024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:06:08.921024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:06:08.924233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.935948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbb976-b6c2-490b-a7f0-80ca9d9d80a2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:08.942947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:08.943601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:08.943965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:08.944243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:08.944923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:08.947029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:08.949630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 779/1585] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:08 2022] PUT /identity/v3/projects/67b7595d39aa4603a78c2f6a2ab7408f/users/e508d401e9284e548470a54b05d91ec7/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:08.950708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:08.996877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 807/1586] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/projects => generated 403 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:08.998411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.040650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de16d6-0cba-490a-8a42-ce0ea91adf5c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.093693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970778f-d352-4d58-8370-93366a7616ae admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.101122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.101849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.102381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.102649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.103833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.106249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.126629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.128219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.128583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.128954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.129778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.132601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.139720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 808/1587] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:09.140967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.164319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d002436-d823-432b-adad-a97f18daa52c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.173031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.173922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.174316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.174790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.175976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.176712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 780/1588] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:08 2022] POST /identity/v3/projects => generated 393 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:09.178118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.178347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.242550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 809/1589] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:09 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:09.243534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.246715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:09.272822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24709e48-f535-47a3-8cb1-5c62da841ae1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.306797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:09.341705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52224387-57fb-4413-ad6e-91a9ea97b655 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.349090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.349659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.350126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.350402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.362976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055ffb7-eab8-4ef0-ba81-334c4d3b17fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.364004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055ffb7-eab8-4ef0-ba81-334c4d3b17fb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.364443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055ffb7-eab8-4ef0-ba81-334c4d3b17fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.364721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055ffb7-eab8-4ef0-ba81-334c4d3b17fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.365471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055ffb7-eab8-4ef0-ba81-334c4d3b17fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.367273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055ffb7-eab8-4ef0-ba81-334c4d3b17fb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.391247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 781/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:09.392614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7dd1d-bdb0-4916-9dce-5e17174eea65 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.393088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7dd1d-bdb0-4916-9dce-5e17174eea65 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.393558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7dd1d-bdb0-4916-9dce-5e17174eea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.444184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:09.447018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dda7dd1d-bdb0-4916-9dce-5e17174eea65 None None] MFA Rules not processed for user `e508d401e9284e548470a54b05d91ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:09.448767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dda7dd1d-bdb0-4916-9dce-5e17174eea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:09.468502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.470303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.483938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 810/1591] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:09.485105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.497018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e68318-f07e-4c2b-a57f-54be53f78f74 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.502912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.503686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.503954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.504221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.504943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.507334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.537411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 811/1592] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:09 2022] POST /identity/v3/users => generated 302 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:09.538611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.551488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218dcfc-638e-44e6-89fb-578b1c1f3930 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.557253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.558201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.558455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.558967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.559628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.561383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.577956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 812/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:09.577956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.592637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74c0b4-cda7-4f1a-b275-8ddd92461b1f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.599232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.600089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.600540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.600906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.601715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.604828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.606309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 782/1594] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:09.607542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7dd1d-bdb0-4916-9dce-5e17174eea65 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.631850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7dd1d-bdb0-4916-9dce-5e17174eea65 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.642376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 813/1595] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:09 2022] POST /identity/v3/users => generated 297 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:09.643705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.648963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.649601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.649974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.650180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.650907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.655138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.671576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f78a51-7236-4c5e-9cf7-024976dfe098 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.680537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.681288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.681472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.681760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] PATH_INFO: `/v3/projects/317ea4e7108a4fc099da6d76a1793ac9/users/ca4779abc6024a0f84f041ca7c058e9a/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.693892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 783/1596] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:09 2022] POST /identity/v3/users => generated 293 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:09.694954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.709043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9f266-b860-4932-b7d5-35aa98326755 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.714482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.715051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=317ea4e7108a4fc099da6d76a1793ac9, user_id=ca4779abc6024a0f84f041ca7c058e9a, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.715302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.715644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.715980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.716732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.719139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.720914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.735143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 784/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:09.736295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.753768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4265a45-9e7a-4ad4-ae75-8be68d2efa7c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.754497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 814/1598] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:09 2022] PUT /identity/v3/projects/317ea4e7108a4fc099da6d76a1793ac9/users/ca4779abc6024a0f84f041ca7c058e9a/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:09.755986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.849891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b699a7-a657-4e05-bb23-8592e03e8c4e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.856334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.857128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.857554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.857908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.861149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.863988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.866522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:09.867154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:09.867433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:09.867712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:09.868356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:09.870005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:09.885938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 785/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:09.886872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.899540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805960af-6c27-4b47-a467-de4815a64a03 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:09.902428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 815/1600] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:09 2022] POST /identity/v3/projects => generated 405 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:09.903662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:09.922618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:10.018214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.018909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.022507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.022507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.022507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.025002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.086221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:10.086221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadb372-f584-46ab-b66b-56bf9bae9c4c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.086990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 786/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:10.086990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.087282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fecdd-e38d-4027-a05b-f09e8adaf037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.087282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fecdd-e38d-4027-a05b-f09e8adaf037 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.087282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fecdd-e38d-4027-a05b-f09e8adaf037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.087282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fecdd-e38d-4027-a05b-f09e8adaf037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.087595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fecdd-e38d-4027-a05b-f09e8adaf037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.087595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fecdd-e38d-4027-a05b-f09e8adaf037 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.087745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc17c16-55c2-49ce-ae86-4b334a5b5748 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.101896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.102611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.102902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.103213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.104103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.104986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 816/1602] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:10.106500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311c1ba-55e3-43b1-90d4-83924c622b8a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.106789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311c1ba-55e3-43b1-90d4-83924c622b8a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.107063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311c1ba-55e3-43b1-90d4-83924c622b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.111556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.143203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 787/1603] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:10.156550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.171928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3311c1ba-55e3-43b1-90d4-83924c622b8a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:10.173191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3311c1ba-55e3-43b1-90d4-83924c622b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:10.179742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ab27c-0b25-4c6a-b654-e882d4f13c96 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.190629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.193936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.194226 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.200216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] PATH_INFO: `/v3/projects/f1959b8e00bc4819a993fdbd0c012f5c/users/5235207aa3b5477eb76215ebb4a43c98/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.201291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 817/1604] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:10.202863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311c1ba-55e3-43b1-90d4-83924c622b8a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.229446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311c1ba-55e3-43b1-90d4-83924c622b8a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.235556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1959b8e00bc4819a993fdbd0c012f5c, user_id=5235207aa3b5477eb76215ebb4a43c98, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.237188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.238507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.238633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.239110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.239528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.240401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.244759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.264020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 788/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:10 2022] PUT /identity/v3/projects/f1959b8e00bc4819a993fdbd0c012f5c/users/5235207aa3b5477eb76215ebb4a43c98/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:10.265578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.297936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 818/1606] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:10 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:10.299173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.317793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba838bf-ce16-4dd9-b439-c11581cfe7a7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.371418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fdda0-668f-44b1-b5df-7caa929cf522 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.378459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.378459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.378459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.378459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.378898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.380311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.395104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 789/1607] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:10.395104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.410810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a175f-865d-4cd7-a070-2382db45561e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.417789 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0982e6-5000-4c58-971d-aacfbc58580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.418415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.419050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.419251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0982e6-5000-4c58-971d-aacfbc58580a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.419364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.419591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.419981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0982e6-5000-4c58-971d-aacfbc58580a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.420247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.421351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0982e6-5000-4c58-971d-aacfbc58580a admin admin] PATH_INFO: `/v3/projects/067762e92ac5441c9dc11c67494f54fa/users/93616015ff4040cab5af7de7489f9071/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.421917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.438167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 819/1608] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:10.439065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.444266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0982e6-5000-4c58-971d-aacfbc58580a admin admin] RBAC: Authorizing `identity:create_grant(project_id=067762e92ac5441c9dc11c67494f54fa, user_id=93616015ff4040cab5af7de7489f9071, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.448230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0982e6-5000-4c58-971d-aacfbc58580a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.472221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 790/1609] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:10 2022] PUT /identity/v3/projects/067762e92ac5441c9dc11c67494f54fa/users/93616015ff4040cab5af7de7489f9071/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:10.474119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06bf74-fb68-40e6-8cda-e28c5e642ff0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.474333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06bf74-fb68-40e6-8cda-e28c5e642ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.475665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06bf74-fb68-40e6-8cda-e28c5e642ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.502381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1f06bf74-fb68-40e6-8cda-e28c5e642ff0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:10.504621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1f06bf74-fb68-40e6-8cda-e28c5e642ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:10.528959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f60c5b-1602-4e74-a944-61ea31a1c8cb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.600477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:10.636560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 791/1610] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:10.637125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06bf74-fb68-40e6-8cda-e28c5e642ff0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.644417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.645784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.646438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.646803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.648692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06bf74-fb68-40e6-8cda-e28c5e642ff0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.653656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.654316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.654578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.654852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.655751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.657321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.659873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.663346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.678196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 820/1611] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:10.679244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.685511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 792/1612] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:10.686501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.704001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2184e-ec37-4015-85ce-448638eedec5 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.708223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd73c-df6e-4503-8e06-b0f1f1384514 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.711645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.712673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.713057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.714032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] PATH_INFO: `/v3/projects/79a06421f9704e9395454e5bda5f4af4/users/0efe1c62e4ea4222b0e1c070670b5af4/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.719408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e54b6-8f39-41af-a04d-06569757b049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.720506 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e54b6-8f39-41af-a04d-06569757b049 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.720879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e54b6-8f39-41af-a04d-06569757b049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.721570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e54b6-8f39-41af-a04d-06569757b049 admin admin] PATH_INFO: `/v3/projects/a47e0b7cb9814ec1885f17bf3c37058b/users/c3c51f18c0f34bddbcd17d984aafc91a/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.755394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a06421f9704e9395454e5bda5f4af4, user_id=0efe1c62e4ea4222b0e1c070670b5af4, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.761494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e54b6-8f39-41af-a04d-06569757b049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47e0b7cb9814ec1885f17bf3c37058b, user_id=c3c51f18c0f34bddbcd17d984aafc91a, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.762961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.767632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e54b6-8f39-41af-a04d-06569757b049 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:10.780567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:10.793373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 793/1613] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:10 2022] PUT /identity/v3/projects/a47e0b7cb9814ec1885f17bf3c37058b/users/c3c51f18c0f34bddbcd17d984aafc91a/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:10.795704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094889f5-d1f3-4d1c-95eb-d5ad4f6057d6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.796197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094889f5-d1f3-4d1c-95eb-d5ad4f6057d6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.796637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094889f5-d1f3-4d1c-95eb-d5ad4f6057d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.797577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 821/1614] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:10 2022] PUT /identity/v3/projects/79a06421f9704e9395454e5bda5f4af4/users/0efe1c62e4ea4222b0e1c070670b5af4/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:10.798602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.842977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-094889f5-d1f3-4d1c-95eb-d5ad4f6057d6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:10.847199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-094889f5-d1f3-4d1c-95eb-d5ad4f6057d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:10.943226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650c428-9eb5-4915-9582-fe5ba8230d14 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:10.948778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16bef2-0a67-4f2c-b5c8-ff4835423a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:10.949511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16bef2-0a67-4f2c-b5c8-ff4835423a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:10.949829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16bef2-0a67-4f2c-b5c8-ff4835423a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:10.950611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16bef2-0a67-4f2c-b5c8-ff4835423a83 admin admin] PATH_INFO: `/v3/projects/db3457ad96e7428a9e5c81385b1adea9/users/14c8ddef7ba64b70aaa635df4218ae28/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:10.960732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 794/1615] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:10.963470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094889f5-d1f3-4d1c-95eb-d5ad4f6057d6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:10.968847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16bef2-0a67-4f2c-b5c8-ff4835423a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3457ad96e7428a9e5c81385b1adea9, user_id=14c8ddef7ba64b70aaa635df4218ae28, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:10.971653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16bef2-0a67-4f2c-b5c8-ff4835423a83 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:11.001865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 822/1616] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:10 2022] PUT /identity/v3/projects/db3457ad96e7428a9e5c81385b1adea9/users/14c8ddef7ba64b70aaa635df4218ae28/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:11.003874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62558-750a-4e5f-866f-6b1b4f32a9a9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.004155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62558-750a-4e5f-866f-6b1b4f32a9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.004425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62558-750a-4e5f-866f-6b1b4f32a9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.063563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dde62558-750a-4e5f-866f-6b1b4f32a9a9 None None] MFA Rules not processed for user `c3c51f18c0f34bddbcd17d984aafc91a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:11.065977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dde62558-750a-4e5f-866f-6b1b4f32a9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:11.087565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094889f5-d1f3-4d1c-95eb-d5ad4f6057d6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:11.177796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:11.180074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.180612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.181040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.195201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 823/1617] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:11.204484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe98207-9e51-425d-a191-40fb3a6f8766 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.205158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe98207-9e51-425d-a191-40fb3a6f8766 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.207667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe98207-9e51-425d-a191-40fb3a6f8766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.263241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:11.329111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4fe98207-9e51-425d-a191-40fb3a6f8766 None None] MFA Rules not processed for user `0efe1c62e4ea4222b0e1c070670b5af4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:11.330598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4fe98207-9e51-425d-a191-40fb3a6f8766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:11.399839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:11.400077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:11.427113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 795/1618] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:11.428010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:11.448403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d32457-c9ce-41e6-9e0a-e5f2142139bb service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:11.454633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec29a5b-3807-4977-97fd-0012aa384274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:11.455484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29a5b-3807-4977-97fd-0012aa384274 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.455778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29a5b-3807-4977-97fd-0012aa384274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.456054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29a5b-3807-4977-97fd-0012aa384274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.457222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec29a5b-3807-4977-97fd-0012aa384274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:11.460945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec29a5b-3807-4977-97fd-0012aa384274 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:11.488272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 796/1619] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:11.488272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1bf19-a561-46fc-93fb-4ab1ad9ba541 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.488272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1bf19-a561-46fc-93fb-4ab1ad9ba541 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.488272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1bf19-a561-46fc-93fb-4ab1ad9ba541 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.493010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 797/1620] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 13:06:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:11.495802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1bf19-a561-46fc-93fb-4ab1ad9ba541 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:11.503647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:11.521240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 824/1621] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:11.521240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe98207-9e51-425d-a191-40fb3a6f8766 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:11.524896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1bf19-a561-46fc-93fb-4ab1ad9ba541 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:11.536806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22169b-cee7-4946-a98c-409266404e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:11.537935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22169b-cee7-4946-a98c-409266404e7c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.537935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22169b-cee7-4946-a98c-409266404e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.538217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22169b-cee7-4946-a98c-409266404e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.548299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe98207-9e51-425d-a191-40fb3a6f8766 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:11.556828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22169b-cee7-4946-a98c-409266404e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:11.558317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22169b-cee7-4946-a98c-409266404e7c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:11.560297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:11.561089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.562168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.562795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] PATH_INFO: `/v3/projects/db3457ad96e7428a9e5c81385b1adea9/users/14c8ddef7ba64b70aaa635df4218ae28/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.570523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3457ad96e7428a9e5c81385b1adea9, user_id=14c8ddef7ba64b70aaa635df4218ae28, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:11.573436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:11.583689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 798/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:11.585019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75affc10-4dd0-46ca-8884-aa9c3519d6e3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.585310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75affc10-4dd0-46ca-8884-aa9c3519d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.585592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75affc10-4dd0-46ca-8884-aa9c3519d6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.617760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 825/1623] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:11 2022] PUT /identity/v3/projects/db3457ad96e7428a9e5c81385b1adea9/users/14c8ddef7ba64b70aaa635df4218ae28/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:11.617760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:11.668592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-75affc10-4dd0-46ca-8884-aa9c3519d6e3 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:11.669890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-75affc10-4dd0-46ca-8884-aa9c3519d6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:11.741756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a32c7f-3e30-439d-a38c-c1ab9d829c81 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:11.751919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9c9cf-bf15-4a66-bada-3ae5806c5e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:11.753000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9c9cf-bf15-4a66-bada-3ae5806c5e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.753375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9c9cf-bf15-4a66-bada-3ae5806c5e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.753766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9c9cf-bf15-4a66-bada-3ae5806c5e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.754499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9c9cf-bf15-4a66-bada-3ae5806c5e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:11.756532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9c9cf-bf15-4a66-bada-3ae5806c5e48 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:11.812401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 826/1624] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:11 2022] POST /identity/v3/projects => generated 393 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:11.813967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac2fb-7e74-48d5-bb61-ce464c911df2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.815449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac2fb-7e74-48d5-bb61-ce464c911df2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.815860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac2fb-7e74-48d5-bb61-ce464c911df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.818026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 799/1625] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:11.818697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75affc10-4dd0-46ca-8884-aa9c3519d6e3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:11.846585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75affc10-4dd0-46ca-8884-aa9c3519d6e3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:11.886631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-68aac2fb-7e74-48d5-bb61-ce464c911df2 None None] MFA Rules not processed for user `14c8ddef7ba64b70aaa635df4218ae28`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:11.888080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-68aac2fb-7e74-48d5-bb61-ce464c911df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:11.961492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:11.962131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:11.962327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:11.962605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:11.963387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:11.965750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.013721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 800/1626] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:11 2022] POST /identity/v3/users => generated 303 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:12.014804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:12.035550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:12.049187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 827/1627] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:12.050266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aac2fb-7e74-48d5-bb61-ce464c911df2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:12.080576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aac2fb-7e74-48d5-bb61-ce464c911df2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:12.095033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:12.095800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.096354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.096821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.097643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:12.099474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.122591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 828/1628] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:12.126495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:12.187111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e690ade-6d5a-4ff5-ae37-bb1eaff38606 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:12.195624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:12.196694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.197145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.197575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.246424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:12.258720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9989c-6bc0-4665-865a-15ad99e090e0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:12.265200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:12.265981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.266410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.266910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.289907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:12.294358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.320477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 829/1629] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:12 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:12.335896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:12.372408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2acf4f-75ec-4c73-9cdd-e7101ab43427 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:12.380763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:12.381801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.382320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.383053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] PATH_INFO: `/v3/projects/bc775c0f00d64e20b6432a89f7210cf2/users/8166e6caf52a4159a9fe2c8a592e6223/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.401968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:12.405544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.426334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc775c0f00d64e20b6432a89f7210cf2, user_id=8166e6caf52a4159a9fe2c8a592e6223, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:12.430349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.462156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 801/1630] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:12.465134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 830/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:12 2022] PUT /identity/v3/projects/bc775c0f00d64e20b6432a89f7210cf2/users/8166e6caf52a4159a9fe2c8a592e6223/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:12.469083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:12.508487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987f2a-9db3-4a66-9d42-07516e1b6af7 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:12.715464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:12.716395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.716742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.717020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.718108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:12.727540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.747662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:12.762312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6738831-de4f-4412-b3e7-1375fd3a896d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:12.770280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:12.771107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.771541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.772012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] PATH_INFO: `/v3/users/e508d401e9284e548470a54b05d91ec7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.773331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e508d401e9284e548470a54b05d91ec7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:12.778623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 802/1632] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:12.780421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:12.780989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.816714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b16cd-7496-4f11-b50a-9f3368403790 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:12.837946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9cd14-0260-4fc3-a08c-e638d9628e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:12.838647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9cd14-0260-4fc3-a08c-e638d9628e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.838980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9cd14-0260-4fc3-a08c-e638d9628e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.839484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9cd14-0260-4fc3-a08c-e638d9628e88 admin admin] PATH_INFO: `/v3/projects/bc775c0f00d64e20b6432a89f7210cf2/users/8166e6caf52a4159a9fe2c8a592e6223/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.844398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9cd14-0260-4fc3-a08c-e638d9628e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc775c0f00d64e20b6432a89f7210cf2, user_id=8166e6caf52a4159a9fe2c8a592e6223, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:12.854767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9cd14-0260-4fc3-a08c-e638d9628e88 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:12.892049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e508d401e9284e548470a54b05d91ec7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:12.895190 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 803/1633] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:12 2022] PUT /identity/v3/projects/bc775c0f00d64e20b6432a89f7210cf2/users/8166e6caf52a4159a9fe2c8a592e6223/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:12.905125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be4709-6df9-43d3-9e2b-1ff8210b5d6e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:12.905485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be4709-6df9-43d3-9e2b-1ff8210b5d6e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:12.905848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be4709-6df9-43d3-9e2b-1ff8210b5d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:12.933446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e508d401e9284e548470a54b05d91ec7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:12.954785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e508d401e9284e548470a54b05d91ec7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:12.956622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e508d401e9284e548470a54b05d91ec7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:12.979226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-927d3815-17c7-4c05-9358-c3b84aba107e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e508d401e9284e548470a54b05d91ec7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:12.992058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-20be4709-6df9-43d3-9e2b-1ff8210b5d6e None None] MFA Rules not processed for user `8166e6caf52a4159a9fe2c8a592e6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:12.993428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-20be4709-6df9-43d3-9e2b-1ff8210b5d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:12.999920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 831/1634] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:12 2022] DELETE /identity/v3/users/e508d401e9284e548470a54b05d91ec7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:13.001369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.001659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.005450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.028115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:13.093983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:13.099131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:13.152480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 804/1635] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:12 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:13.154596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a34798-fd3d-43b4-aeee-863c4624bcd9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.155102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a34798-fd3d-43b4-aeee-863c4624bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.155580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a34798-fd3d-43b4-aeee-863c4624bcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.242179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f4a34798-fd3d-43b4-aeee-863c4624bcd9 None None] MFA Rules not processed for user `93616015ff4040cab5af7de7489f9071`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:13.243606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f4a34798-fd3d-43b4-aeee-863c4624bcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:13.326996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 832/1636] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:13.328176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:13.329641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:13.392653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 805/1637] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:13 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:13.395095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545d76-403f-49d3-b0f6-aa5b9f812b22 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.395256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545d76-403f-49d3-b0f6-aa5b9f812b22 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.395480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545d76-403f-49d3-b0f6-aa5b9f812b22 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.397131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 806/1638] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 13:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:13.397927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64545d76-403f-49d3-b0f6-aa5b9f812b22 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:13.399628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-64545d76-403f-49d3-b0f6-aa5b9f812b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:13.448347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:13.491036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64545d76-403f-49d3-b0f6-aa5b9f812b22 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:13.495341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:13.496020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.496134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc98ec-f15b-4a46-a401-e22568385283 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:13.496344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.496575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.512403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fa07c-3250-4066-a5c4-d836a20f7ffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:13.513068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa07c-3250-4066-a5c4-d836a20f7ffc service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.517008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa07c-3250-4066-a5c4-d836a20f7ffc service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.517354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa07c-3250-4066-a5c4-d836a20f7ffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.527296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:13.529017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:13.535902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fa07c-3250-4066-a5c4-d836a20f7ffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:13.537407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fa07c-3250-4066-a5c4-d836a20f7ffc service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:13.606720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 807/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:13.607066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:13.610064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 833/1640] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:13.610916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:13.612686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08163f01-867a-431e-b52a-08d145a1b568 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.614490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08163f01-867a-431e-b52a-08d145a1b568 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.621174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08163f01-867a-431e-b52a-08d145a1b568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.673645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c28a5d-cd35-466d-867b-e697c880b403 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:13.680333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:13.681033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.681588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.681915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.682569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:13.686749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:13.690762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-08163f01-867a-431e-b52a-08d145a1b568 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:13.692171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-08163f01-867a-431e-b52a-08d145a1b568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:13.760207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 808/1641] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:13 2022] POST /identity/v3/projects => generated 403 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:13.761304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:13.764257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:13.878536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:13.882083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 834/1642] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:06:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:13.883190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08163f01-867a-431e-b52a-08d145a1b568 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:13.915119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08163f01-867a-431e-b52a-08d145a1b568 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:13.948400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944429-72ce-49e7-9f22-abf765291f51 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:13.958814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:13.959748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:13.960194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:13.960645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:13.962884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:14.067479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:14.068190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:14.068602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:14.068979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:14.069786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:14.071981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:14.109369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 835/1643] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:13 2022] POST /identity/v3/users => generated 308 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:14.110464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:14.135112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:14.137941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464f310-750e-4c89-87e8-4f18ce976813 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:14.147150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:14.148505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:14.149092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:14.152238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:14.157053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:14.183613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 809/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:14.185082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:14.204068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304743e-cb41-4939-8945-94ce062a1787 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:14.211374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:14.212036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:14.212312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:14.212585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:14.213313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:14.221942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:14.259100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 810/1645] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:14.264087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:14.301353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899baa3-30e1-4fc6-b984-ca1cc679b395 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:14.308936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:14.316578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:14.317203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:14.317486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:14.318036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] PATH_INFO: `/v3/projects/91f55be365914b4cb44b75ffec529b1d/users/7efd36c6b1a44c798e040a473ea2817f/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:14.344222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:14.345571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f55be365914b4cb44b75ffec529b1d, user_id=7efd36c6b1a44c798e040a473ea2817f, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:14.345804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:14.348703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:14.371420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 811/1646] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:14 2022] PUT /identity/v3/projects/91f55be365914b4cb44b75ffec529b1d/users/7efd36c6b1a44c798e040a473ea2817f/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:14.372844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:14.375020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 836/1647] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:14.527463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43054e02-7dd0-44ce-a93e-28cf0c631091 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:14.539867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:14.540432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:14.540715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:14.541041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:14.541716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:14.543375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:14.563110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 812/1648] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:14.567784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:14.597412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd482ee-488b-48b5-a0ff-0a2d33f2248e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:14.634593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:14.635702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:14.636313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:14.636607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] PATH_INFO: `/v3/projects/91f55be365914b4cb44b75ffec529b1d/users/7efd36c6b1a44c798e040a473ea2817f/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:14.641037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f55be365914b4cb44b75ffec529b1d, user_id=7efd36c6b1a44c798e040a473ea2817f, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:14.647290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:14.690119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 837/1649] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:14 2022] PUT /identity/v3/projects/91f55be365914b4cb44b75ffec529b1d/users/7efd36c6b1a44c798e040a473ea2817f/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:15.439286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:15.441800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:15.487993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0690091-a9be-40a7-9f23-ae1e453bc69a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:15.650973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:15.739833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:15.740726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:15.741059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:15.741440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] PATH_INFO: `/v3/projects/67b7595d39aa4603a78c2f6a2ab7408f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:15.743574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b7595d39aa4603a78c2f6a2ab7408f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:15.746455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:15.921518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b7595d39aa4603a78c2f6a2ab7408f', 'request_id': 'req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:15.936012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b7595d39aa4603a78c2f6a2ab7408f', 'request_id': 'req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:15.957786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b7595d39aa4603a78c2f6a2ab7408f', 'request_id': 'req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:15.980811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] The token cache is being invalidate because project 67b7595d39aa4603a78c2f6a2ab7408f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:15.983108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:15.984354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2b79865-2d01-4dc3-b97e-2d56f3e7019f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:15.992540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 813/1650] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:15 2022] DELETE /identity/v3/projects/67b7595d39aa4603a78c2f6a2ab7408f => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:15.994505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:15.999478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:16.043432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927a081-c7eb-4c5c-bf0f-c0c7582f477f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:16.189653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:16.190881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:16.191410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:16.191910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] PATH_INFO: `/v3/users/14c8ddef7ba64b70aaa635df4218ae28` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:16.194635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c8ddef7ba64b70aaa635df4218ae28)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:16.197274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:16.223230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824e02-b8f4-44fb-a393-4d53fcf31239 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:16.223545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824e02-b8f4-44fb-a393-4d53fcf31239 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:16.223821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824e02-b8f4-44fb-a393-4d53fcf31239 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:16.225659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 814/1651] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 13:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:06:16.247233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38351f82-8280-4622-8e0a-45cdd40e4e95 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:16.247689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38351f82-8280-4622-8e0a-45cdd40e4e95 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:16.248049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38351f82-8280-4622-8e0a-45cdd40e4e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:16.293504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-38351f82-8280-4622-8e0a-45cdd40e4e95 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:16.295989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-38351f82-8280-4622-8e0a-45cdd40e4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:16.323765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c8ddef7ba64b70aaa635df4218ae28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:16.362674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c8ddef7ba64b70aaa635df4218ae28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:16.387872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c8ddef7ba64b70aaa635df4218ae28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:16.388704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c8ddef7ba64b70aaa635df4218ae28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:16.412726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f45840b3-db10-4d0f-b928-094d638dab37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c8ddef7ba64b70aaa635df4218ae28'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:16.417289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 815/1652] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 13:06:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:16.421456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85d178-072b-4aa3-b353-9e18d466e032 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:16.422277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85d178-072b-4aa3-b353-9e18d466e032 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:16.422793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85d178-072b-4aa3-b353-9e18d466e032 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:16.424607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 816/1653] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 13:06:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:16.432190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 838/1654] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:15 2022] DELETE /identity/v3/users/14c8ddef7ba64b70aaa635df4218ae28 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:16.433070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:16.433070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:16.433230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:16.467902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:16.469155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:16.530696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:16.582304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 839/1655] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:06:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:16.599501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85d178-072b-4aa3-b353-9e18d466e032 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:16.601290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8a85d178-072b-4aa3-b353-9e18d466e032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:16.709473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85d178-072b-4aa3-b353-9e18d466e032 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:16.718100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926cfb3-417e-43ea-b7fc-64359b7be610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:16.719221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926cfb3-417e-43ea-b7fc-64359b7be610 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:16.719916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926cfb3-417e-43ea-b7fc-64359b7be610 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:16.720430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926cfb3-417e-43ea-b7fc-64359b7be610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:16.757095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926cfb3-417e-43ea-b7fc-64359b7be610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:16.762029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926cfb3-417e-43ea-b7fc-64359b7be610 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:16.793741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 817/1656] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:17.078333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:17.079044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:17.214747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a37e85-bd4a-4c1f-b81e-878e743a0515 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:17.221685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:17.221685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:17.222002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:17.222245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] PATH_INFO: `/v3/projects/db3457ad96e7428a9e5c81385b1adea9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:17.223783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3457ad96e7428a9e5c81385b1adea9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:17.225705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:06:17.225705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Sep 07 13:06:17.225705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: Sep 07 13:06:17.227480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:17.309913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3457ad96e7428a9e5c81385b1adea9', 'request_id': 'req-382a7295-8dc1-45cb-8a49-b75d45512173'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:17.326531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3457ad96e7428a9e5c81385b1adea9', 'request_id': 'req-382a7295-8dc1-45cb-8a49-b75d45512173'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:17.338431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3457ad96e7428a9e5c81385b1adea9', 'request_id': 'req-382a7295-8dc1-45cb-8a49-b75d45512173'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:17.355553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] The token cache is being invalidate because project db3457ad96e7428a9e5c81385b1adea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:17.355956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:17.357184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-382a7295-8dc1-45cb-8a49-b75d45512173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:17.361723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 840/1657] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:17 2022] DELETE /identity/v3/projects/db3457ad96e7428a9e5c81385b1adea9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:17.366234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55906b87-dcdf-49f2-a960-8f1dde35b3fe None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:17.366864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55906b87-dcdf-49f2-a960-8f1dde35b3fe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:17.368146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55906b87-dcdf-49f2-a960-8f1dde35b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:17.387645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:17.416628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-55906b87-dcdf-49f2-a960-8f1dde35b3fe None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:17.418027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-55906b87-dcdf-49f2-a960-8f1dde35b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:17.546749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 818/1658] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:17.548441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0fde8-768f-4c51-9b81-468fdd1c73d2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:17.548868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0fde8-768f-4c51-9b81-468fdd1c73d2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:17.549382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0fde8-768f-4c51-9b81-468fdd1c73d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:17.581812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-90c0fde8-768f-4c51-9b81-468fdd1c73d2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:17.583819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90c0fde8-768f-4c51-9b81-468fdd1c73d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:17.613031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 841/1659] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:17.619128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55906b87-dcdf-49f2-a960-8f1dde35b3fe None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:17.632529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55906b87-dcdf-49f2-a960-8f1dde35b3fe None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:17.640722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:17.641763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:17.642427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:17.643033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:17.644749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:17.645266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:17.648003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:17.668152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 819/1660] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:17.669236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0fde8-768f-4c51-9b81-468fdd1c73d2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:17.686087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0fde8-768f-4c51-9b81-468fdd1c73d2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:17.692152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5604c-3455-4e5e-be02-192d1e3b22f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:17.693005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5604c-3455-4e5e-be02-192d1e3b22f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:17.693417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5604c-3455-4e5e-be02-192d1e3b22f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:17.693944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5604c-3455-4e5e-be02-192d1e3b22f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:17.694729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5604c-3455-4e5e-be02-192d1e3b22f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:17.697188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5604c-3455-4e5e-be02-192d1e3b22f7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:17.737897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 842/1661] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:17 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:17.742310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:17.755769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48ccf3-4379-4811-ad0e-246d8184e093 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:17.756233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48ccf3-4379-4811-ad0e-246d8184e093 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:17.756520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48ccf3-4379-4811-ad0e-246d8184e093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:17.758521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480f51-ce5e-48ee-9d91-a87fc8baf726 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:17.834969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fb48ccf3-4379-4811-ad0e-246d8184e093 None None] MFA Rules not processed for user `c0db39216fd24b5f9cb0531bcedb0326`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:17.837279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fb48ccf3-4379-4811-ad0e-246d8184e093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:17.912739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:17.914292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:17.915005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:17.915740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:17.917240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:17.920278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:17.926422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:17.985782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 820/1662] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:17 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:17.992315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:18.010800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 843/1663] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:18.012309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.012603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.012883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.035931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9f8d0-678c-4b8a-932b-93ddac693fe6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:18.054252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:18.054887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.055165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.055483 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.056135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:18.064533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:18.070967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] MFA Rules not processed for user `c0db39216fd24b5f9cb0531bcedb0326`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:18.072827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:18.107600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 821/1664] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:18.108661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:18.111626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 844/1665] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:18.125410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:18.127222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:18.150364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2835d-22d3-4141-829a-e6f80f390f27 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:18.175249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54196023-d039-4c16-99f6-32db771c38ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:18.175925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196023-d039-4c16-99f6-32db771c38ee admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.176205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196023-d039-4c16-99f6-32db771c38ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.176478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196023-d039-4c16-99f6-32db771c38ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.177136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54196023-d039-4c16-99f6-32db771c38ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:18.178849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54196023-d039-4c16-99f6-32db771c38ee admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:18.218524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 822/1666] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:18.219708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcaae7-5ff0-415a-8d5c-9275bec64fed None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.219996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcaae7-5ff0-415a-8d5c-9275bec64fed None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.220272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcaae7-5ff0-415a-8d5c-9275bec64fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.324778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ff87c-c9bc-4320-96f7-1d30f529b484 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:18.335735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4bdcaae7-5ff0-415a-8d5c-9275bec64fed None None] MFA Rules not processed for user `7efd36c6b1a44c798e040a473ea2817f`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:18.336995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4bdcaae7-5ff0-415a-8d5c-9275bec64fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:18.345256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:18.346197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.346602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.346952 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.364610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:18.366523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:18.389385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 845/1667] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:18.390689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:18.422625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629c13a-2e75-4184-9ddb-4e2ef3701137 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:18.444234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:18.445111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.445563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.446134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] PATH_INFO: `/v3/projects/329819bc48c242df813cc4e36faa88e0/users/5a70fbdf54c24e46b7c81008808808f1/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.475571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:18.503778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329819bc48c242df813cc4e36faa88e0, user_id=5a70fbdf54c24e46b7c81008808808f1, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:18.506784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:18.536903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 823/1668] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:18.545289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 846/1669] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:18 2022] PUT /identity/v3/projects/329819bc48c242df813cc4e36faa88e0/users/5a70fbdf54c24e46b7c81008808808f1/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:18.555707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70101912-7b63-4a6f-9590-705b69725d3f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.556731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70101912-7b63-4a6f-9590-705b69725d3f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.557016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70101912-7b63-4a6f-9590-705b69725d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.651633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-70101912-7b63-4a6f-9590-705b69725d3f None None] MFA Rules not processed for user `5a70fbdf54c24e46b7c81008808808f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:18.654222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-70101912-7b63-4a6f-9590-705b69725d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:18.827769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:18.828175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 824/1670] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:18.960672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1cf47-a133-4d3b-822d-9ab10f1e81c2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:18.977847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:18.978638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:18.979052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:18.979522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:18.980363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:18.982999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:19.053827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 847/1671] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:18 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:06:19.057932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70101912-7b63-4a6f-9590-705b69725d3f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:19.081954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70101912-7b63-4a6f-9590-705b69725d3f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:19.167079 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:19.168552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.168976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.169349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.170175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:19.172236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:19.223337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 825/1672] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:19 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:19.224536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:19.240479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c89ed-d5c5-46da-a9ab-a93cafdb0cf6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:19.249038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:19.250358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.261685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.265164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.268390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:19.271964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:19.308140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:19.315530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 848/1673] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:06:19.319919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63862f-dca7-460b-989f-8b5e248e5e1d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:19.325557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:19.326644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.327158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.327778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] PATH_INFO: `/v3/projects/cbecfc3e7497400eac1c0e9066ed5c5f/users/b0f37bbc0f3345d1994074d9b938aacc/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.346809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbecfc3e7497400eac1c0e9066ed5c5f, user_id=b0f37bbc0f3345d1994074d9b938aacc, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:19.350311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:19.374051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 826/1674] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:19 2022] PUT /identity/v3/projects/cbecfc3e7497400eac1c0e9066ed5c5f/users/b0f37bbc0f3345d1994074d9b938aacc/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:06:19.382341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:19.400145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:19.408042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0d96b-9e22-43a8-b12f-f1045af7a1fa admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:19.542802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a1d3d-8dba-4b2e-942e-4821bdce2607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:19.543545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a1d3d-8dba-4b2e-942e-4821bdce2607 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.543825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a1d3d-8dba-4b2e-942e-4821bdce2607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.544091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a1d3d-8dba-4b2e-942e-4821bdce2607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.544740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a1d3d-8dba-4b2e-942e-4821bdce2607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:19.547013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a1d3d-8dba-4b2e-942e-4821bdce2607 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:19.561138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 849/1675] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:19.562784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:19.576914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aefeb4-9b67-4fe7-9e1f-750dcd26044e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:19.582520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:19.583325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.583681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.584069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] PATH_INFO: `/v3/projects/cbecfc3e7497400eac1c0e9066ed5c5f/users/b0f37bbc0f3345d1994074d9b938aacc/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.586338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbecfc3e7497400eac1c0e9066ed5c5f, user_id=b0f37bbc0f3345d1994074d9b938aacc, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:19.589870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:19.616806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 827/1676] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:19 2022] PUT /identity/v3/projects/cbecfc3e7497400eac1c0e9066ed5c5f/users/b0f37bbc0f3345d1994074d9b938aacc/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:19.621472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b573-f911-4a03-8b1e-180fb7867468 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.622092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b573-f911-4a03-8b1e-180fb7867468 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.622402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b573-f911-4a03-8b1e-180fb7867468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.664001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a884b573-f911-4a03-8b1e-180fb7867468 None None] MFA Rules not processed for user `b0f37bbc0f3345d1994074d9b938aacc`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:19.665430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a884b573-f911-4a03-8b1e-180fb7867468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:19.772245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 850/1677] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:19.792343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:19.795130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:19.834713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:19.843224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d2f2b-7e62-45e5-9146-9d2cb5eb71e4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.843744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d2f2b-7e62-45e5-9146-9d2cb5eb71e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.844252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d2f2b-7e62-45e5-9146-9d2cb5eb71e4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.846279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 851/1678] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 13:06:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:06:19.853975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab187e-23af-4920-93ea-c2654db75699 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.854552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab187e-23af-4920-93ea-c2654db75699 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.854929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab187e-23af-4920-93ea-c2654db75699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:19.962121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f359f-ce2c-4a82-acd5-94b10a7bf44b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:19.966322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7bab187e-23af-4920-93ea-c2654db75699 None None] MFA Rules not processed for user `4302574365514eb1b25bd8e62736a4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:19.970523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7bab187e-23af-4920-93ea-c2654db75699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:19.972515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:19.973336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:19.974262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:19.974366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:20.005988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:20.007570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:20.045683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 828/1679] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:20.113644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 852/1680] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:06:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:20.150557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:20.152536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:20.186407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d8bdf-3363-4865-8daa-17d091117f0c service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:20.197454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cf3d9-8f20-4141-adca-fab616bce61e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:20.198137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf3d9-8f20-4141-adca-fab616bce61e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:20.198472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf3d9-8f20-4141-adca-fab616bce61e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:20.198746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cf3d9-8f20-4141-adca-fab616bce61e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:20.224666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cf3d9-8f20-4141-adca-fab616bce61e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:20.226254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cf3d9-8f20-4141-adca-fab616bce61e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:20.247126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 829/1681] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:20.293853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab187e-23af-4920-93ea-c2654db75699 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:20.323597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab187e-23af-4920-93ea-c2654db75699 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:20.342272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e599d-e994-4f6d-be96-288d9d906bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:20.342914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e599d-e994-4f6d-be96-288d9d906bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:20.343194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e599d-e994-4f6d-be96-288d9d906bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:20.343462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e599d-e994-4f6d-be96-288d9d906bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:20.469758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:20.587842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e599d-e994-4f6d-be96-288d9d906bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:20.589782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e599d-e994-4f6d-be96-288d9d906bb0 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:20.618093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 853/1682] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:21.856906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:21.857453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:21.857854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:21.932419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] MFA Rules not processed for user `3cad6d025da54eb29d09fd2a2888c844`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:21.934097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:22.042135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 830/1683] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:22.056293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf785c-f732-4b4c-8c81-d308035df5b7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:22.056868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf785c-f732-4b4c-8c81-d308035df5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:22.060645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf785c-f732-4b4c-8c81-d308035df5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:22.108738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a7cf785c-f732-4b4c-8c81-d308035df5b7 None None] MFA Rules not processed for user `3cad6d025da54eb29d09fd2a2888c844`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:22.110301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7cf785c-f732-4b4c-8c81-d308035df5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:22.123277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 854/1684] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:22.141829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:22.143935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:22.196227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd85c5c-8fd4-438b-b235-ed92e765b61f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:22.196883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd85c5c-8fd4-438b-b235-ed92e765b61f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:22.197427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd85c5c-8fd4-438b-b235-ed92e765b61f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:22.199475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 855/1685] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:06:22.273559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902e8bc-94da-41f6-b815-f46ee132c9d8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:22.282009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57361-1974-4b04-bf0a-ee9368c52aea None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:22.282371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:22.282993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:22.283270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:22.283659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:22.284049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57361-1974-4b04-bf0a-ee9368c52aea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:22.286924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57361-1974-4b04-bf0a-ee9368c52aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:22.308643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:22.310785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:22.324492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 831/1686] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:22.341513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e0e57361-1974-4b04-bf0a-ee9368c52aea None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:22.343864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e0e57361-1974-4b04-bf0a-ee9368c52aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:22.686750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 856/1687] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:06:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:22.718427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:22.720634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:22.833756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171d7f9-31df-45c7-9406-6a31ba2036fd service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:22.860965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dd225-3978-43c4-b12d-5d0c2a2be89d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:22.865307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd225-3978-43c4-b12d-5d0c2a2be89d service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:22.865865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd225-3978-43c4-b12d-5d0c2a2be89d service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:22.866354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd225-3978-43c4-b12d-5d0c2a2be89d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:22.898607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dd225-3978-43c4-b12d-5d0c2a2be89d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:22.909879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dd225-3978-43c4-b12d-5d0c2a2be89d service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:22.964543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 832/1688] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:23.004103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5d3fe-0ff6-45c6-9b93-636b9fbcecb0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:23.004391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5d3fe-0ff6-45c6-9b93-636b9fbcecb0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:23.004674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5d3fe-0ff6-45c6-9b93-636b9fbcecb0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:23.007413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 857/1689] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 13:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:23.013355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d063c-f4f7-447a-9bfe-50846bb146f2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:23.013355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d063c-f4f7-447a-9bfe-50846bb146f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:23.014005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d063c-f4f7-447a-9bfe-50846bb146f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:23.049710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-990d063c-f4f7-447a-9bfe-50846bb146f2 None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:23.059838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-990d063c-f4f7-447a-9bfe-50846bb146f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:23.077603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 833/1690] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 13:06:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:23.085906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5d3fe-0ff6-45c6-9b93-636b9fbcecb0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:23.118531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5d3fe-0ff6-45c6-9b93-636b9fbcecb0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:23.134514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a439d9f-6826-4b57-9898-b77e62e6a086 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:23.135129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439d9f-6826-4b57-9898-b77e62e6a086 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:23.135446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439d9f-6826-4b57-9898-b77e62e6a086 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:23.135777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439d9f-6826-4b57-9898-b77e62e6a086 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:23.144276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4a439d9f-6826-4b57-9898-b77e62e6a086 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:23.187163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a439d9f-6826-4b57-9898-b77e62e6a086 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:23.188914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a439d9f-6826-4b57-9898-b77e62e6a086 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:23.235041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 858/1691] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:23.478750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cd831-4766-4926-a9b6-b58011d723f6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:23.479268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cd831-4766-4926-a9b6-b58011d723f6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:23.479722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cd831-4766-4926-a9b6-b58011d723f6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:23.482342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 834/1692] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 13:06:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:23.489859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7678b2e-471f-45a0-9da4-a18f12ca7c90 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:23.490372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7678b2e-471f-45a0-9da4-a18f12ca7c90 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:23.490802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7678b2e-471f-45a0-9da4-a18f12ca7c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:23.524579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d7678b2e-471f-45a0-9da4-a18f12ca7c90 None None] MFA Rules not processed for user `4302574365514eb1b25bd8e62736a4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:23.528135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7678b2e-471f-45a0-9da4-a18f12ca7c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:23.545871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 859/1693] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:06:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:23.577549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cd831-4766-4926-a9b6-b58011d723f6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:23.579351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8a5cd831-4766-4926-a9b6-b58011d723f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:23.626975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cd831-4766-4926-a9b6-b58011d723f6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:23.635666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:23.636521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:23.636926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:23.637204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:23.654508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:23.662075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:23.664939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:23.699964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 835/1694] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:24.634808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:24.635412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:24.635412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:24.665077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] MFA Rules not processed for user `3cad6d025da54eb29d09fd2a2888c844`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:24.671251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:24.694855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 860/1695] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:24.899741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:24.901976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:25.131705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d31476-d50a-4f33-bce8-cadd52248644 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:25.140432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:25.143783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:25.144073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:25.144341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] PATH_INFO: `/v3/users/93616015ff4040cab5af7de7489f9071` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:25.150506 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] RBAC: Authorizing `identity:delete_user(user_id=93616015ff4040cab5af7de7489f9071)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:25.151863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:06:25.151863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Sep 07 13:06:25.151863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: Sep 07 13:06:25.153344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:25.458201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93616015ff4040cab5af7de7489f9071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:25.482571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93616015ff4040cab5af7de7489f9071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:25.539975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93616015ff4040cab5af7de7489f9071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:25.542107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93616015ff4040cab5af7de7489f9071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:25.590867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93616015ff4040cab5af7de7489f9071'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:25.629258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 836/1696] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:24 2022] DELETE /identity/v3/users/93616015ff4040cab5af7de7489f9071 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:26.974924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:26.977050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:27.013504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a2919-50a7-4931-811a-42a5d7e603ba None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:27.180485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:27.181939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:27.301604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:27.302324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:27.306898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:27.308554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] PATH_INFO: `/v3/users/0efe1c62e4ea4222b0e1c070670b5af4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:27.311319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efe1c62e4ea4222b0e1c070670b5af4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:27.313813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:27.418558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:27.450326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dd708-60ce-46c8-b23b-9013dffc895b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:27.466911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efe1c62e4ea4222b0e1c070670b5af4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.471554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:27.472170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:27.472458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:27.472739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] PATH_INFO: `/v3/projects/067762e92ac5441c9dc11c67494f54fa` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:27.476934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=067762e92ac5441c9dc11c67494f54fa)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:27.485421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:27.506433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efe1c62e4ea4222b0e1c070670b5af4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.531991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efe1c62e4ea4222b0e1c070670b5af4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.533046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efe1c62e4ea4222b0e1c070670b5af4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.547195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:27.564977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5ef02b3b-27ee-4ec1-a716-9f6c13aa84c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efe1c62e4ea4222b0e1c070670b5af4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.585842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 861/1697] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:26 2022] DELETE /identity/v3/users/0efe1c62e4ea4222b0e1c070670b5af4 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:27.604577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:27.608624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:27.613991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:27.674840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067762e92ac5441c9dc11c67494f54fa', 'request_id': 'req-4987b9a2-67cd-431d-afaf-3ad3909fa555'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.699808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067762e92ac5441c9dc11c67494f54fa', 'request_id': 'req-4987b9a2-67cd-431d-afaf-3ad3909fa555'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.738808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067762e92ac5441c9dc11c67494f54fa', 'request_id': 'req-4987b9a2-67cd-431d-afaf-3ad3909fa555'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.817472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] The token cache is being invalidate because project 067762e92ac5441c9dc11c67494f54fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:27.817807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.833700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4987b9a2-67cd-431d-afaf-3ad3909fa555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:27.839980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 837/1698] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:27 2022] DELETE /identity/v3/projects/067762e92ac5441c9dc11c67494f54fa => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:27.844600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:27.844874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:27.845154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:27.977443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:27.982885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:27.992386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:27.993656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:28.288772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 838/1699] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:28.292895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:28.295333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:28.310658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 862/1700] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 720 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:28.329967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4e0a-bde0-4f58-9725-33ea9626e623 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:28.352873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:28.352873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:28.352873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:28.353441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] PATH_INFO: `/v3/users/7efd36c6b1a44c798e040a473ea2817f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:28.359735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7efd36c6b1a44c798e040a473ea2817f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:28.361845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:28.457227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:28.463060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:28.565340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7efd36c6b1a44c798e040a473ea2817f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:28.605757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572719f9-e080-49fc-b4a3-84f2bc2aca78 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:28.607933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7efd36c6b1a44c798e040a473ea2817f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:28.623475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:28.624111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:28.624390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:28.624658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:28.649572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:28.660274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7efd36c6b1a44c798e040a473ea2817f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:28.661364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7efd36c6b1a44c798e040a473ea2817f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:28.670654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:28.679292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7efd36c6b1a44c798e040a473ea2817f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:28.695025 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 839/1701] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:28 2022] DELETE /identity/v3/users/7efd36c6b1a44c798e040a473ea2817f => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:28.696190 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:28.703517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:28.727374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 863/1702] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:28.764234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:28.931744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7c419-6146-4246-afe5-78891668d041 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:28.942181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:28.943374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:28.943805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:28.944211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:28.951686 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:29.114910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:29.116563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:29.169862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 840/1703] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:06:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:06:29.778325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:29.778805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:29.800927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f390-1c47-4563-8a97-c87b6a1a31ae service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:29.827148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:29.901739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:29.903411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:29.903902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:29.904527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] PATH_INFO: `/v3/users/c3c51f18c0f34bddbcd17d984aafc91a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:29.906268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c51f18c0f34bddbcd17d984aafc91a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:29.908416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:29.952306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:29.954399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:29.979062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc1841-e230-4447-a39f-9032e23e18c2 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:29.990405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:29.991364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:29.991791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:29.992206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] PATH_INFO: `/v3/projects/91f55be365914b4cb44b75ffec529b1d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:29.994404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f55be365914b4cb44b75ffec529b1d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:29.995340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c51f18c0f34bddbcd17d984aafc91a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:29.996950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:30.034604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c51f18c0f34bddbcd17d984aafc91a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.052994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c51f18c0f34bddbcd17d984aafc91a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.053886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c51f18c0f34bddbcd17d984aafc91a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.067924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c43b26e-f2e2-4b20-9413-a14b6484b80d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c51f18c0f34bddbcd17d984aafc91a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.081504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 864/1704] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:29 2022] DELETE /identity/v3/users/c3c51f18c0f34bddbcd17d984aafc91a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:30.086404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a2d0d-d7a9-4fd4-a48c-1db32daf79fb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.086695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a2d0d-d7a9-4fd4-a48c-1db32daf79fb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.086977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a2d0d-d7a9-4fd4-a48c-1db32daf79fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.144948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4f2a2d0d-d7a9-4fd4-a48c-1db32daf79fb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:30.147421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f2a2d0d-d7a9-4fd4-a48c-1db32daf79fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:30.147888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f55be365914b4cb44b75ffec529b1d', 'request_id': 'req-46f65ebb-8a36-4c93-80bd-36231708125c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.160409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f55be365914b4cb44b75ffec529b1d', 'request_id': 'req-46f65ebb-8a36-4c93-80bd-36231708125c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.198930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f55be365914b4cb44b75ffec529b1d', 'request_id': 'req-46f65ebb-8a36-4c93-80bd-36231708125c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.219138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] The token cache is being invalidate because project 91f55be365914b4cb44b75ffec529b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:30.220823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.225030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-46f65ebb-8a36-4c93-80bd-36231708125c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:30.230790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 841/1705] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:29 2022] DELETE /identity/v3/projects/91f55be365914b4cb44b75ffec529b1d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:30.235514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.236244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.237394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.283292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:30.285181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:30.328316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 865/1706] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:30.328515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:30.389277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 842/1707] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:30.404621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f94c63-8311-46ff-a202-9c236df8f9d1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.405088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f94c63-8311-46ff-a202-9c236df8f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.405582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f94c63-8311-46ff-a202-9c236df8f9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.410903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:30.412359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:30.477363 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-07f94c63-8311-46ff-a202-9c236df8f9d1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:30.479164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-07f94c63-8311-46ff-a202-9c236df8f9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:30.494389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 866/1708] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:30.495510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f94c63-8311-46ff-a202-9c236df8f9d1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:30.509424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f94c63-8311-46ff-a202-9c236df8f9d1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:30.515647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:30.516358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.516634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.517454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.518240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:30.518548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:30.520315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:30.549699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 867/1709] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:30.553469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:30.560448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886712d-8616-45db-9530-f4d2c0663468 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:30.567019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:30.568303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.568303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.568303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.580723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecb34-fc5b-4ac5-8123-d5d81ffca63d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:30.593873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:30.594408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.594611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:30.594745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.595020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.595682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:30.596197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:30.598807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:30.618646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 843/1710] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:06:30.622358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:30.659608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:30.660161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 868/1711] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:30 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:30.673548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab75e4-064b-4763-aff1-c00d3d7155be service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:30.851174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:30.852290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.852683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.853575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.855003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:30.858826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:30.904513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:30.918103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 844/1712] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:30 2022] POST /identity/v3/users => generated 292 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:30.922086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83a26-5e2a-499e-be16-85908bfa7658 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:30.930863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:30.932586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:30.932716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:30.932910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:30.933916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:30.937438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:30.977715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 869/1713] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:30.983424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:31.001701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c32356-028a-4bfb-8317-232e1b59282c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:31.029639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:31.031249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:31.032806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:31.034021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:31.036676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:31.042549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:31.078745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:31.080955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:31.105521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 845/1714] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:31.106735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:31.119961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe03b-d3af-4430-8a19-d81f27ee9c6c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:31.124241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fcca7-18ee-41c6-a88c-be84c826ee89 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:31.133543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:31.134552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:31.135072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:31.136362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] PATH_INFO: `/v3/projects/79a06421f9704e9395454e5bda5f4af4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:31.138637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a06421f9704e9395454e5bda5f4af4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:31.139290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5c2ec-2231-4777-bc23-e7a912277b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:31.140435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5c2ec-2231-4777-bc23-e7a912277b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:31.141288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5c2ec-2231-4777-bc23-e7a912277b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:31.141724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5c2ec-2231-4777-bc23-e7a912277b0e admin admin] PATH_INFO: `/v3/projects/c5d67c52ee8947f3b7d1b4b6a81db146/users/b49f70a3bf8e4a4cbc89b3020c02e6a4/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:31.142715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:31.197581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5c2ec-2231-4777-bc23-e7a912277b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d67c52ee8947f3b7d1b4b6a81db146, user_id=b49f70a3bf8e4a4cbc89b3020c02e6a4, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:31.200876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5c2ec-2231-4777-bc23-e7a912277b0e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:31.238774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 846/1715] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:31 2022] PUT /identity/v3/projects/c5d67c52ee8947f3b7d1b4b6a81db146/users/b49f70a3bf8e4a4cbc89b3020c02e6a4/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:31.260267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:31.260799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:31.261246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:31.328908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a06421f9704e9395454e5bda5f4af4', 'request_id': 'req-0ed28051-114b-45cb-9389-cfb56a870183'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.354268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] MFA Rules not processed for user `b49f70a3bf8e4a4cbc89b3020c02e6a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:31.355903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:31.362023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a06421f9704e9395454e5bda5f4af4', 'request_id': 'req-0ed28051-114b-45cb-9389-cfb56a870183'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.382191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:31.431007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a06421f9704e9395454e5bda5f4af4', 'request_id': 'req-0ed28051-114b-45cb-9389-cfb56a870183'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.462460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] The token cache is being invalidate because project 79a06421f9704e9395454e5bda5f4af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:31.467126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.468788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.471733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 870/1716] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:31 2022] DELETE /identity/v3/projects/79a06421f9704e9395454e5bda5f4af4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:31.477299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:31.480151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:31.579807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 847/1717] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:31.585908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:31.588546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:31.621813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28051-114b-45cb-9389-cfb56a870183 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:31.630570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:31.631890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:31.632313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:31.632871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] PATH_INFO: `/v3/users/8166e6caf52a4159a9fe2c8a592e6223` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:31.634963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8166e6caf52a4159a9fe2c8a592e6223)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:31.640354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:31.704094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074388da-9105-4e48-aa36-fd0faf94c873 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:31.724009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:31.724718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:31.728356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:31.728747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:31.737121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:31.743411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:31.754645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8166e6caf52a4159a9fe2c8a592e6223'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.810178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8166e6caf52a4159a9fe2c8a592e6223'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.820901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:31.833927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8166e6caf52a4159a9fe2c8a592e6223'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.838737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8166e6caf52a4159a9fe2c8a592e6223'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.867984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 848/1718] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:31 2022] POST /identity/v3/projects => generated 387 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:31.874233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:31.874825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8166e6caf52a4159a9fe2c8a592e6223'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:31.881025 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:31.887780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 871/1719] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:31 2022] DELETE /identity/v3/users/8166e6caf52a4159a9fe2c8a592e6223 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:31.983592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084af8c3-005c-4762-b9ee-4d0b524b993b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:32.052230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:32.053006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.053291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.053565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.054744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:32.062303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:32.103338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 849/1720] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:31 2022] POST /identity/v3/users => generated 300 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:32.110033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.128658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ca43b-fa1f-4aac-948f-feedb7f49aec admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:32.135698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:32.136333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.136895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.137190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.138593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:32.140796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:32.161766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 872/1721] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:32.163026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.173208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13ec82-e79b-4676-b3c4-950cfa86c042 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:32.179339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde28249-2ba2-4059-a7e1-23950bbfaa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:32.180088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde28249-2ba2-4059-a7e1-23950bbfaa91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.180382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde28249-2ba2-4059-a7e1-23950bbfaa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.180656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde28249-2ba2-4059-a7e1-23950bbfaa91 admin admin] PATH_INFO: `/v3/projects/6aeece72312942659057f61876e09854/users/c34782d8722441c89b90ff0f80e00ad0/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.210755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde28249-2ba2-4059-a7e1-23950bbfaa91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aeece72312942659057f61876e09854, user_id=c34782d8722441c89b90ff0f80e00ad0, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:32.214053 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde28249-2ba2-4059-a7e1-23950bbfaa91 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:32.230505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:32.238596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 850/1722] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:32 2022] PUT /identity/v3/projects/6aeece72312942659057f61876e09854/users/c34782d8722441c89b90ff0f80e00ad0/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:32.244275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.260026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c5cc5-4dae-4e4f-b507-2a847bba712d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:32.370500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:32.372125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.372616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.389362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.389875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:32.391860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:32.414823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab41d-4192-4c32-a929-f6bb71776eca None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.415330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab41d-4192-4c32-a929-f6bb71776eca None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.415441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab41d-4192-4c32-a929-f6bb71776eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.462414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 873/1723] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:32.463555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.471361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:32.531953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4d9ab41d-4192-4c32-a929-f6bb71776eca None None] MFA Rules not processed for user `ca4779abc6024a0f84f041ca7c058e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:32.533247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4d9ab41d-4192-4c32-a929-f6bb71776eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:32.540224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab74fb-4856-4b88-9b07-fdeefc9801b8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:32.561175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:32.562618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.563703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.563995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] PATH_INFO: `/v3/projects/a47e0b7cb9814ec1885f17bf3c37058b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.569659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47e0b7cb9814ec1885f17bf3c37058b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:32.572193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:32.706267 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47e0b7cb9814ec1885f17bf3c37058b', 'request_id': 'req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:32.719584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47e0b7cb9814ec1885f17bf3c37058b', 'request_id': 'req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:32.751027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47e0b7cb9814ec1885f17bf3c37058b', 'request_id': 'req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:32.771310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 851/1724] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:32.775494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab41d-4192-4c32-a929-f6bb71776eca None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.799871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] The token cache is being invalidate because project a47e0b7cb9814ec1885f17bf3c37058b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:32.800175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:32.801444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:32.807022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 874/1725] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:32 2022] DELETE /identity/v3/projects/a47e0b7cb9814ec1885f17bf3c37058b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:32.807022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.807022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:32.814365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab41d-4192-4c32-a929-f6bb71776eca None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:32.917909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:32.918163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.918376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.918663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] PATH_INFO: `/v3/projects/6aeece72312942659057f61876e09854/users/c34782d8722441c89b90ff0f80e00ad0/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.928947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aeece72312942659057f61876e09854, user_id=c34782d8722441c89b90ff0f80e00ad0, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:32.937871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:32.947890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:32.974366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 852/1726] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:32 2022] PUT /identity/v3/projects/6aeece72312942659057f61876e09854/users/c34782d8722441c89b90ff0f80e00ad0/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:32.975291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.985243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e7e26-dd5b-4808-93d9-ba07fcd89ae0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:32.988060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:33.108393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5a1cd-fc9d-4340-98d7-a08d2bb38d84 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:33.130355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa282bd-7ed2-4fb6-a6ef-cfa4b30b3f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:33.131014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa282bd-7ed2-4fb6-a6ef-cfa4b30b3f47 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:33.131303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa282bd-7ed2-4fb6-a6ef-cfa4b30b3f47 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:33.131574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa282bd-7ed2-4fb6-a6ef-cfa4b30b3f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:33.135396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aaa282bd-7ed2-4fb6-a6ef-cfa4b30b3f47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:33.164844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:33.240195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa282bd-7ed2-4fb6-a6ef-cfa4b30b3f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:33.242183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa282bd-7ed2-4fb6-a6ef-cfa4b30b3f47 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:33.266051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 853/1727] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:33.267774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054a409-212d-42d1-a972-4faf73a7462e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:33.268264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054a409-212d-42d1-a972-4faf73a7462e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:33.268733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054a409-212d-42d1-a972-4faf73a7462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:33.292732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a054a409-212d-42d1-a972-4faf73a7462e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:33.294649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a054a409-212d-42d1-a972-4faf73a7462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:33.337431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:33.338354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:33.338633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:33.338904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] PATH_INFO: `/v3/projects/bc775c0f00d64e20b6432a89f7210cf2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:33.351771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc775c0f00d64e20b6432a89f7210cf2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:33.364421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:33.501002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 854/1728] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:33.506144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8975de7-1c26-41b6-8817-1a22d7badcd2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:33.506812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8975de7-1c26-41b6-8817-1a22d7badcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:33.507414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8975de7-1c26-41b6-8817-1a22d7badcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:33.620127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a8975de7-1c26-41b6-8817-1a22d7badcd2 None None] MFA Rules not processed for user `c34782d8722441c89b90ff0f80e00ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:33.628289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a8975de7-1c26-41b6-8817-1a22d7badcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:33.635703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc775c0f00d64e20b6432a89f7210cf2', 'request_id': 'req-cd1c8428-41af-46eb-b861-cd00c2ad915b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:33.654095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc775c0f00d64e20b6432a89f7210cf2', 'request_id': 'req-cd1c8428-41af-46eb-b861-cd00c2ad915b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:33.689283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc775c0f00d64e20b6432a89f7210cf2', 'request_id': 'req-cd1c8428-41af-46eb-b861-cd00c2ad915b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:33.739245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] The token cache is being invalidate because project bc775c0f00d64e20b6432a89f7210cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:33.739546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:33.741196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cd1c8428-41af-46eb-b861-cd00c2ad915b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:33.759628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 875/1729] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:32 2022] DELETE /identity/v3/projects/bc775c0f00d64e20b6432a89f7210cf2 => generated 0 bytes in 952 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:33.759628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2364-f1a5-461f-a9a9-401ef5cd9faa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:33.759628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2364-f1a5-461f-a9a9-401ef5cd9faa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:33.759628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2364-f1a5-461f-a9a9-401ef5cd9faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:33.819616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-98fd2364-f1a5-461f-a9a9-401ef5cd9faa None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:33.820887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98fd2364-f1a5-461f-a9a9-401ef5cd9faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:33.879864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 855/1730] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:33.881310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:33.881550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:33.881860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:33.968640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] MFA Rules not processed for user `4302574365514eb1b25bd8e62736a4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:33.969929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:34.062540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 876/1731] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:34.063848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9f87f-99e3-4c58-98ee-411ca1e5b22f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.064137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9f87f-99e3-4c58-98ee-411ca1e5b22f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.064411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9f87f-99e3-4c58-98ee-411ca1e5b22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.115777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:34.132612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ace9f87f-99e3-4c58-98ee-411ca1e5b22f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:34.133909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ace9f87f-99e3-4c58-98ee-411ca1e5b22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:34.160218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 856/1732] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Sep 7 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:34.160942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:34.171523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:34.178054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 877/1733] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:34.178769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9f87f-99e3-4c58-98ee-411ca1e5b22f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:34.211404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9f87f-99e3-4c58-98ee-411ca1e5b22f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:34.226034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a1d19-a268-4d75-8c09-56d21c332c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:34.226570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a1d19-a268-4d75-8c09-56d21c332c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.226858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a1d19-a268-4d75-8c09-56d21c332c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.227126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a1d19-a268-4d75-8c09-56d21c332c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.227778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a1d19-a268-4d75-8c09-56d21c332c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:34.228112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a1d19-a268-4d75-8c09-56d21c332c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:34.229823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a1d19-a268-4d75-8c09-56d21c332c73 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:34.253091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 878/1734] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:34.253955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.254243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.254518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.357697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:34.367392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75c856-0210-4c1d-9995-f316e5fe2b4d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:34.383466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] MFA Rules not processed for user `5235207aa3b5477eb76215ebb4a43c98`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:34.384678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:34.386941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a229c4-7744-4a6f-a763-33f0632e8376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:34.387600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a229c4-7744-4a6f-a763-33f0632e8376 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.387861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a229c4-7744-4a6f-a763-33f0632e8376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.388143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a229c4-7744-4a6f-a763-33f0632e8376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.432587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a229c4-7744-4a6f-a763-33f0632e8376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:34.457170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a229c4-7744-4a6f-a763-33f0632e8376 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:34.473767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 857/1735] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:34.484360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.484652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.484928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.577070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:34.578347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:34.619498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 858/1736] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:34.620359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:34.623727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:34.658776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320153f-934b-482c-9a5a-3cca1c193084 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:34.680898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:34.681522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.681834 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.682112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.710088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:34.711478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:34.766494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 859/1737] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:34.771175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 879/1738] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:34.774068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:34.778574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:34.779766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:34.860968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8aafc-c9d0-4746-a9da-23974d84943b service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:34.875076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb8c7-0f7c-4175-9dbf-ebbc23d88ae7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:34.902328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:34.905152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:34.906289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.906775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.907197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.910675 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:34.911041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:34.911330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:34.912044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:34.920659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:34.948911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:34.950711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:34.986929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 880/1739] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:34.990082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:35.010775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:35.011061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 860/1740] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:34 2022] POST /identity/v3/projects => generated 371 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:35.012063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:35.036950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb2afc-02af-4c60-b882-d7c26414b709 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:35.198027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:35.203530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:35.204210 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:35.204553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] PATH_INFO: `/v3/users/b49f70a3bf8e4a4cbc89b3020c02e6a4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:35.209329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49f70a3bf8e4a4cbc89b3020c02e6a4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:35.216037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:35.272461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f20060-7938-4dec-bdb6-6734bc5523e9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:35.293296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:35.294155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:35.294595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:35.295012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:35.296389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:35.298800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:35.304888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:35.362605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 861/1741] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:35.363751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:35.370670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:35.453643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49f70a3bf8e4a4cbc89b3020c02e6a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:35.531900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49f70a3bf8e4a4cbc89b3020c02e6a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:35.572667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49f70a3bf8e4a4cbc89b3020c02e6a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:35.576017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49f70a3bf8e4a4cbc89b3020c02e6a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:35.600025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49f70a3bf8e4a4cbc89b3020c02e6a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:35.619862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:35.634039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 881/1742] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:34 2022] DELETE /identity/v3/users/b49f70a3bf8e4a4cbc89b3020c02e6a4 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:35.635244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:35.637081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:35.651148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:35.765023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12574a14-aadd-409a-b2f8-2ff8ccd50749 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:35.780080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647bb9b-bf62-46f7-9bc9-2b1919fdd731 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:35.791747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:35.792657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:35.792947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:35.793225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:35.793959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:35.797054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:35.840682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 882/1743] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:35 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:35.841947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:35.846012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:35.860963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12576a65-6c4a-43e9-b261-a676ace56049 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:35.866235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cd192-1bcd-41f3-9cf5-21d961f19a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:35.867000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cd192-1bcd-41f3-9cf5-21d961f19a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:35.875164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cd192-1bcd-41f3-9cf5-21d961f19a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:35.875418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cd192-1bcd-41f3-9cf5-21d961f19a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:35.887932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cd192-1bcd-41f3-9cf5-21d961f19a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:35.890539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cd192-1bcd-41f3-9cf5-21d961f19a72 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:35.955762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 883/1744] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:35 2022] POST /identity/v3/projects => generated 407 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:35.959551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b514ff-6177-454e-9c2b-dee8bc73bd0c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:35.962526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b514ff-6177-454e-9c2b-dee8bc73bd0c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:35.962944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b514ff-6177-454e-9c2b-dee8bc73bd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.019993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-19b514ff-6177-454e-9c2b-dee8bc73bd0c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:36.024105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-19b514ff-6177-454e-9c2b-dee8bc73bd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:36.086230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.087056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.087391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.098017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.102516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:36.295038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.296784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:36.316472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 884/1745] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:36.317889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b514ff-6177-454e-9c2b-dee8bc73bd0c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:36.326952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 862/1746] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 964 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:36.328990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:36.356769 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b514ff-6177-454e-9c2b-dee8bc73bd0c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:36.358141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758eb6b-d2ca-4a44-8e65-66b3e6294192 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:36.369945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.370897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.371326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.371745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.372824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.391057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.391057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.391057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.391057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.391445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.391445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:36.404338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:36.418539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 885/1747] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:36.422791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:36.472044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e26276-98ae-4803-8599-85944602cba5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:36.472716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 863/1748] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:36 2022] POST /identity/v3/users => generated 304 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:36.478779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:36.490962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.491747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.492256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.492540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.493281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.495592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:36.509328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720667e-a7bf-4a64-a25d-af33e806754f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:36.532912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 886/1749] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:36.534196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:36.547606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.548235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.548517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.548794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.549559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.551165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:36.571173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0c70-315d-4924-b1c7-6487d3f0dd24 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:36.590141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ce530-2d9b-4a1c-b0d3-e417fb0518c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.593566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ce530-2d9b-4a1c-b0d3-e417fb0518c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.598115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ce530-2d9b-4a1c-b0d3-e417fb0518c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.599437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ce530-2d9b-4a1c-b0d3-e417fb0518c2 admin admin] PATH_INFO: `/v3/projects/2ccbf25b6bbc4116b461e5c49c69bda1/users/b56c77cb57d54036a60fdff39959098e/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.606403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 864/1750] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:36.607408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:36.641166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ce530-2d9b-4a1c-b0d3-e417fb0518c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccbf25b6bbc4116b461e5c49c69bda1, user_id=b56c77cb57d54036a60fdff39959098e, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.645100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ce530-2d9b-4a1c-b0d3-e417fb0518c2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:36.645715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cba3d1-0561-420c-8482-2d78b7fc73cd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:36.672945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.673597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.673908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.674213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.674879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.677454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:36.699974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 887/1751] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:36 2022] PUT /identity/v3/projects/2ccbf25b6bbc4116b461e5c49c69bda1/users/b56c77cb57d54036a60fdff39959098e/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:06:36.702296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.702732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.703076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.728096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 865/1752] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:36.732688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:36.764492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c277c-ec28-476b-a154-9485d64131e4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:36.803089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] MFA Rules not processed for user `b56c77cb57d54036a60fdff39959098e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:36.804539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:36.851900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:36.906944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:36.940015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:36.940926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.941102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.941505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] PATH_INFO: `/v3/projects/673fd028eadd4bc380ec8a50e2b3465f/users/de1322b26ff94e2399716ff38d78c476/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.987915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=673fd028eadd4bc380ec8a50e2b3465f, user_id=de1322b26ff94e2399716ff38d78c476, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:36.990753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:37.011843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 888/1753] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:37.013043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.019974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:37.028481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 866/1754] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:36 2022] PUT /identity/v3/projects/673fd028eadd4bc380ec8a50e2b3465f/users/de1322b26ff94e2399716ff38d78c476/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:37.054124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.101994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a18dc-236b-49cf-a2f1-7dab117db1e4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:37.185063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712d990-80da-4d27-9957-c8a5f67718da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:37.186380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712d990-80da-4d27-9957-c8a5f67718da service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:37.186955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712d990-80da-4d27-9957-c8a5f67718da service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:37.187793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712d990-80da-4d27-9957-c8a5f67718da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:37.227269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306cd3d-a4c5-4cfe-a25f-4867240e3c7d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:37.244429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:37.245252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:37.245553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:37.245701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:37.251991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:37.254153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:37.281482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712d990-80da-4d27-9957-c8a5f67718da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:37.283126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712d990-80da-4d27-9957-c8a5f67718da service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:37.297305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 889/1755] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:37.299754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa46e89-10fe-4c8c-93b3-80f1f37af33a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:37.300367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa46e89-10fe-4c8c-93b3-80f1f37af33a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:37.300367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa46e89-10fe-4c8c-93b3-80f1f37af33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:37.352205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 867/1756] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:37 2022] POST /identity/v3/projects => generated 379 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:37.356187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.435069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8aa46e89-10fe-4c8c-93b3-80f1f37af33a None None] MFA Rules not processed for user `de1322b26ff94e2399716ff38d78c476`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:37.446188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8aa46e89-10fe-4c8c-93b3-80f1f37af33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:37.483380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:37.610903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c796c20-682d-42c8-bfac-f9b202b691e9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:37.628959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 890/1757] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:37 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:37.631629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:37.632937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa46e89-10fe-4c8c-93b3-80f1f37af33a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.634950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:37.635454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:37.635969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:37.637861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:37.647897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:37.687049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa46e89-10fe-4c8c-93b3-80f1f37af33a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:37.705279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:37.707883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:37.708673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:37.710272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:37.714453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:37.718197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:37.741695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 868/1758] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:37 2022] POST /identity/v3/users => generated 296 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:37.745091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.800945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8d709-48a9-4c1f-85d3-27171d9b9d7a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:37.812358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:37.814361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:37.815070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:37.815590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:37.820960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:37.831126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:37.852633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 891/1759] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:37 2022] POST /identity/v3/projects => generated 407 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:37.855184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.939304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 869/1760] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:37.943278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.998840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ddde4-6ab3-4d21-a4bf-318010030175 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:38.116512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d60450-b4d9-4aac-9380-c3684c1255a8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:38.130948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:38.131692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:38.131978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:38.132254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:38.134060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:38.139297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:38.203131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 892/1761] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:37 2022] POST /identity/v3/users => generated 310 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:38.207407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:38.210075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:38.264359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:38.265112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:38.265722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:38.266039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] PATH_INFO: `/v3/projects/a65c24e216724bf5960cc427960e883c/users/dc30a2ab5003454c8b0171026aecb091/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:38.285761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65c24e216724bf5960cc427960e883c, user_id=dc30a2ab5003454c8b0171026aecb091, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:38.289843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b38e8-c129-49c5-8d34-27a763cd45de admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:38.296394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:38.333779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:38.341431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:38.345440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:38.345902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:38.350411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:38.358367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 870/1762] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:37 2022] PUT /identity/v3/projects/a65c24e216724bf5960cc427960e883c/users/dc30a2ab5003454c8b0171026aecb091/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:38.364837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:38.367810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:38.436756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 893/1763] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:38.442511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:38.506015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5a4f-6e53-48de-9bb1-816c19d016ef admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:38.555349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1780c0-7a8e-4be6-b1f6-425cabfd708c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:38.561847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:38.562381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:38.562878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:38.563299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:38.564390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:38.565945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:38.595027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 871/1764] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:38.610200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:38.652419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc738ad-b027-4118-99e4-5dc2a5c6dc1a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:38.675896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827f5d7-0eaa-447e-ab42-ee69a942a16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:38.676780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827f5d7-0eaa-447e-ab42-ee69a942a16a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:38.677065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827f5d7-0eaa-447e-ab42-ee69a942a16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:38.677415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827f5d7-0eaa-447e-ab42-ee69a942a16a admin admin] PATH_INFO: `/v3/projects/a65c24e216724bf5960cc427960e883c/users/dc30a2ab5003454c8b0171026aecb091/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:38.694516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827f5d7-0eaa-447e-ab42-ee69a942a16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65c24e216724bf5960cc427960e883c, user_id=dc30a2ab5003454c8b0171026aecb091, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:38.698407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827f5d7-0eaa-447e-ab42-ee69a942a16a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:38.797460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 872/1765] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:38 2022] PUT /identity/v3/projects/a65c24e216724bf5960cc427960e883c/users/dc30a2ab5003454c8b0171026aecb091/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:38.800994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:38.801292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:38.801576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:38.826854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:38.828764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:38.829207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:38.830904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] PATH_INFO: `/v3/projects/0e9c73a5accc4310b163fee52fdb9251/users/2ca72ace2a174a9dba959f8139251958/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:38.909283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9c73a5accc4310b163fee52fdb9251, user_id=2ca72ace2a174a9dba959f8139251958, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:38.917572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:38.917661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] MFA Rules not processed for user `dc30a2ab5003454c8b0171026aecb091`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:38.918854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:38.928821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:38.967963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 894/1766] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:38 2022] PUT /identity/v3/projects/0e9c73a5accc4310b163fee52fdb9251/users/2ca72ace2a174a9dba959f8139251958/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:38.969592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:39.178515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dab60f-255d-4be5-90e7-443176975c41 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:39.200240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:39.200907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:39.205197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:39.205197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:39.205197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:39.205197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:39.217837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 873/1767] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:39.232431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:39.234222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:39.248020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 895/1768] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:39.249043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:39.266047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181ee40-217e-4f96-8127-fbb17538018a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:39.279139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c3241-b8fc-4092-bc97-c05af14973ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:39.279863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3241-b8fc-4092-bc97-c05af14973ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:39.280357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3241-b8fc-4092-bc97-c05af14973ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:39.280775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3241-b8fc-4092-bc97-c05af14973ef admin admin] PATH_INFO: `/v3/projects/0e9c73a5accc4310b163fee52fdb9251/users/2ca72ace2a174a9dba959f8139251958/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:39.290127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3241-b8fc-4092-bc97-c05af14973ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9c73a5accc4310b163fee52fdb9251, user_id=2ca72ace2a174a9dba959f8139251958, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:39.307804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3241-b8fc-4092-bc97-c05af14973ef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:39.348019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:39.372140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 896/1769] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:39 2022] PUT /identity/v3/projects/0e9c73a5accc4310b163fee52fdb9251/users/2ca72ace2a174a9dba959f8139251958/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:39.374233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:39.374597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:39.374951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:39.453808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] MFA Rules not processed for user `2ca72ace2a174a9dba959f8139251958`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:39.457610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:39.555733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194782f-ae7a-482b-bf5b-9b28692416cf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:39.792878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 897/1770] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:39 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:39.839996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:39.840705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:39.841930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:39.842214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:39.892695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:39.896111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:39.935923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae897411-76ce-41c3-8ec6-650ac0b6e56e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:39.959226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:39.959949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:39.960322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:39.960563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:39.982312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:39.984168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:39.997194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 898/1771] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:40.065339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:40.067549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:40.118656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 874/1772] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 890 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:06:40.543475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:40.555709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2380f-8e7b-4c3a-aa91-2831bd68d7ea service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:40.567225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:40.568093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a29dc-664b-46da-b87e-888ac722c6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:40.568722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a29dc-664b-46da-b87e-888ac722c6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:40.569004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a29dc-664b-46da-b87e-888ac722c6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:40.569277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a29dc-664b-46da-b87e-888ac722c6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:40.586562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939ba6f-ad22-4112-895c-2e64b08a3797 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:40.602449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d1160-3eb1-49eb-ac5e-333a88c12fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:40.606976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1160-3eb1-49eb-ac5e-333a88c12fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:40.607257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1160-3eb1-49eb-ac5e-333a88c12fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:40.608786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1160-3eb1-49eb-ac5e-333a88c12fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:40.726728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:40.751040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a29dc-664b-46da-b87e-888ac722c6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:40.752636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a29dc-664b-46da-b87e-888ac722c6b4 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:40.788918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 899/1773] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:40.847431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d1160-3eb1-49eb-ac5e-333a88c12fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:40.849447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d1160-3eb1-49eb-ac5e-333a88c12fd3 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:40.880689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 875/1774] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:40 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:06:41.293711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73167cf-eaf3-4d0d-ab02-bfc6d1590e03 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:41.294747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73167cf-eaf3-4d0d-ab02-bfc6d1590e03 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:41.295237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73167cf-eaf3-4d0d-ab02-bfc6d1590e03 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:41.296879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 900/1775] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:06:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:41.378793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:41.380071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:41.380570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:41.442850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:41.448514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:41.565262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:41.732085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 876/1776] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:41.905633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73167cf-eaf3-4d0d-ab02-bfc6d1590e03 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:41.910453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c73167cf-eaf3-4d0d-ab02-bfc6d1590e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:41.956483 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:41.959350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:41.964553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73167cf-eaf3-4d0d-ab02-bfc6d1590e03 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:41.989065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:41.990512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:41.993216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:41.994101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:42.037806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:42.039909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:42.106782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 901/1777] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:42.181356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df896f-fd84-4232-bcc9-72533884ed59 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:42.196247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:42.197442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:42.198024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:42.198643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] PATH_INFO: `/v3/projects/c5d67c52ee8947f3b7d1b4b6a81db146` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:42.200866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d67c52ee8947f3b7d1b4b6a81db146)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:42.203334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:42.335319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d67c52ee8947f3b7d1b4b6a81db146', 'request_id': 'req-bcfb0e80-671c-4289-99e5-4fc2b3526577'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.372143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d67c52ee8947f3b7d1b4b6a81db146', 'request_id': 'req-bcfb0e80-671c-4289-99e5-4fc2b3526577'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.409342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d67c52ee8947f3b7d1b4b6a81db146', 'request_id': 'req-bcfb0e80-671c-4289-99e5-4fc2b3526577'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.443767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] The token cache is being invalidate because project c5d67c52ee8947f3b7d1b4b6a81db146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:42.444260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.445385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcfb0e80-671c-4289-99e5-4fc2b3526577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.449466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 877/1778] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:41 2022] DELETE /identity/v3/projects/c5d67c52ee8947f3b7d1b4b6a81db146 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:42.453758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:42.479146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82f24-e5ae-4c65-b1dc-76d7597192dc service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:42.716521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:42.717309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:42.719654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:42.720053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] PATH_INFO: `/v3/users/c34782d8722441c89b90ff0f80e00ad0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:42.722672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34782d8722441c89b90ff0f80e00ad0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:42.726991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:42.857606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34782d8722441c89b90ff0f80e00ad0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.904482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34782d8722441c89b90ff0f80e00ad0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.926264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34782d8722441c89b90ff0f80e00ad0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.931866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34782d8722441c89b90ff0f80e00ad0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.933321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db023-c6d8-4eb9-9e5d-7f2f7228077a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:42.935538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db023-c6d8-4eb9-9e5d-7f2f7228077a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:42.935538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db023-c6d8-4eb9-9e5d-7f2f7228077a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:42.935810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 878/1779] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:42.953989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34782d8722441c89b90ff0f80e00ad0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:42.985793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 902/1780] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:42 2022] DELETE /identity/v3/users/c34782d8722441c89b90ff0f80e00ad0 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:43.045623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1c1d-8711-4933-b099-1bfc3423443b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:43.046286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1c1d-8711-4933-b099-1bfc3423443b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:43.048209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1c1d-8711-4933-b099-1bfc3423443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:43.110396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4fba1c1d-8711-4933-b099-1bfc3423443b None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:43.126627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4fba1c1d-8711-4933-b099-1bfc3423443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:43.353860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:43.459564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 879/1781] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:43.591216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:43.599305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:43.715934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cade0c0-4f30-4411-975b-911759d6ce62 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:43.745970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb39191-b66e-4005-b4ce-ee92347f5a65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:43.746656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39191-b66e-4005-b4ce-ee92347f5a65 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:43.746997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39191-b66e-4005-b4ce-ee92347f5a65 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:43.747327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39191-b66e-4005-b4ce-ee92347f5a65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:43.789861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb39191-b66e-4005-b4ce-ee92347f5a65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:43.801532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb39191-b66e-4005-b4ce-ee92347f5a65 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:43.835276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 903/1782] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:44.476997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:44.479697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:44.480346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:44.555009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] MFA Rules not processed for user `ca4779abc6024a0f84f041ca7c058e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:44.558245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:44.911094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 880/1783] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:46.128018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320e85-3f6b-458e-9281-7b94ae88d593 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:46.128573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320e85-3f6b-458e-9281-7b94ae88d593 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:46.128886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320e85-3f6b-458e-9281-7b94ae88d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:46.163466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fa320e85-3f6b-458e-9281-7b94ae88d593 None None] MFA Rules not processed for user `5235207aa3b5477eb76215ebb4a43c98`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:46.165137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa320e85-3f6b-458e-9281-7b94ae88d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:46.234317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:46.275184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 904/1784] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:46.400457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:46.401952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:46.523673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11b82f-a5ce-4586-8c0d-931cd5ab10df None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:46.535983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:46.536653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:46.537098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:46.537382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] PATH_INFO: `/v3/projects/6aeece72312942659057f61876e09854` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:46.538647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aeece72312942659057f61876e09854)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:46.540826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:46.618088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aeece72312942659057f61876e09854', 'request_id': 'req-4a7b549d-02c6-449b-9105-47f62dfe8678'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:46.626810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aeece72312942659057f61876e09854', 'request_id': 'req-4a7b549d-02c6-449b-9105-47f62dfe8678'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:46.636394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aeece72312942659057f61876e09854', 'request_id': 'req-4a7b549d-02c6-449b-9105-47f62dfe8678'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:46.655345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] The token cache is being invalidate because project 6aeece72312942659057f61876e09854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:46.655932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:46.656846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4a7b549d-02c6-449b-9105-47f62dfe8678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:46.659040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 881/1785] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:46 2022] DELETE /identity/v3/projects/6aeece72312942659057f61876e09854 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:46.664190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537eeffd-c66a-4a64-8551-983c844ffff4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:46.664477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537eeffd-c66a-4a64-8551-983c844ffff4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:46.664782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537eeffd-c66a-4a64-8551-983c844ffff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:46.720037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-537eeffd-c66a-4a64-8551-983c844ffff4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:46.725893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-537eeffd-c66a-4a64-8551-983c844ffff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:46.827378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 905/1786] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:46.837865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cef06-bea5-4d9b-9586-d18a56c317f1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:46.838353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cef06-bea5-4d9b-9586-d18a56c317f1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:46.838785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cef06-bea5-4d9b-9586-d18a56c317f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:46.870951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-361cef06-bea5-4d9b-9586-d18a56c317f1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:46.872605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-361cef06-bea5-4d9b-9586-d18a56c317f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:46.882173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 882/1787] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:46.887022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537eeffd-c66a-4a64-8551-983c844ffff4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:46.899764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537eeffd-c66a-4a64-8551-983c844ffff4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:46.904758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:46.905439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:46.905878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:46.906451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:46.907249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:06:46.907613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:46.911741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:46.930184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 906/1788] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:06:46.935144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cef06-bea5-4d9b-9586-d18a56c317f1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:46.945456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cef06-bea5-4d9b-9586-d18a56c317f1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:46.950917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:46.951160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:46.951582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:46.951995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:46.952915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:46.954999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:46.965367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:46.998536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 883/1789] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:46 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:47.001707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.020857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1084ef1-4fe2-4991-a5a1-e9d31af5a745 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.155669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.156387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.156785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.157127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.157975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.160487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.204161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 907/1790] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:47 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:47.209822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.223788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c032-44c7-4593-a05f-81f67cf769e8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.230265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.230974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.231528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.231890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.232887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.235009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.252038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 884/1791] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:47.253309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.264783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bffe1-b027-458b-ac33-ee816f3345d9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.276386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880a270-e270-4c42-89fb-914005d0db47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.284168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a270-e270-4c42-89fb-914005d0db47 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.284380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a270-e270-4c42-89fb-914005d0db47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.287714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a270-e270-4c42-89fb-914005d0db47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.288503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880a270-e270-4c42-89fb-914005d0db47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.290863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880a270-e270-4c42-89fb-914005d0db47 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.314885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 908/1792] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:06:47.320085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.339259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d2133-eeca-47ef-bec3-f5e788da53cd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.350353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.351430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.351930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.352283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] PATH_INFO: `/v3/projects/02a1c9a43be54723a335ecfcc7bb2b77/users/1519c5416bfa4443827d0295d0e6e706/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.376812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a1c9a43be54723a335ecfcc7bb2b77, user_id=1519c5416bfa4443827d0295d0e6e706, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.381520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.414642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 885/1793] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:47 2022] PUT /identity/v3/projects/02a1c9a43be54723a335ecfcc7bb2b77/users/1519c5416bfa4443827d0295d0e6e706/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:47.425740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac59f1d-ee2a-4880-8528-5949346b9713 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.426515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac59f1d-ee2a-4880-8528-5949346b9713 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.426802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac59f1d-ee2a-4880-8528-5949346b9713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.475148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cac59f1d-ee2a-4880-8528-5949346b9713 None None] MFA Rules not processed for user `1519c5416bfa4443827d0295d0e6e706`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:47.476732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cac59f1d-ee2a-4880-8528-5949346b9713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:47.551361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:47.585568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 909/1794] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:47.586634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.675750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a8b71-8bf0-4d2b-a67e-b9651ed51313 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.680483 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.681098 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.681376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.682007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.682320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.684213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.715554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 886/1795] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:47 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:47.719717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac59f1d-ee2a-4880-8528-5949346b9713 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.732413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac59f1d-ee2a-4880-8528-5949346b9713 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.801956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.802604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.802893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.803328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.804352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.806873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.829999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 910/1796] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:47 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:47.835854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.849568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6bbe-1846-410c-89ed-877993bef1b4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.855570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.856197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.856473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.856744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.857422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.859095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.871805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 887/1797] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:47.876488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.886943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c159739-021c-4f37-b475-fe5c3cb94a1d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.891875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc42ea1-ab7e-4df6-8388-a2d9b726d893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.892483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc42ea1-ab7e-4df6-8388-a2d9b726d893 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.892761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc42ea1-ab7e-4df6-8388-a2d9b726d893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.893080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc42ea1-ab7e-4df6-8388-a2d9b726d893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.893893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc42ea1-ab7e-4df6-8388-a2d9b726d893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.895522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc42ea1-ab7e-4df6-8388-a2d9b726d893 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.907009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 911/1798] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:47.911509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:47.922654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa889f-ebe6-4afb-ba24-a782bd7c1f9a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:47.929087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:47.929786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.930071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.930379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] PATH_INFO: `/v3/projects/db88b32425314286972645f72db7aec7/users/5a2d5a370e3d480bbecf81a0378b2319/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:47.947046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db88b32425314286972645f72db7aec7, user_id=5a2d5a370e3d480bbecf81a0378b2319, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:47.950444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:47.966873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 888/1799] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:47 2022] PUT /identity/v3/projects/db88b32425314286972645f72db7aec7/users/5a2d5a370e3d480bbecf81a0378b2319/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:47.979208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69da84-a552-42ed-bd03-58e793243622 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:47.979500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69da84-a552-42ed-bd03-58e793243622 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:47.979813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69da84-a552-42ed-bd03-58e793243622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.010494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ae69da84-a552-42ed-bd03-58e793243622 None None] MFA Rules not processed for user `5a2d5a370e3d480bbecf81a0378b2319`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:48.011708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae69da84-a552-42ed-bd03-58e793243622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:48.090513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 912/1800] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:48.095388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.170437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819892c8-fc71-4106-a68b-d48d98b1e12a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.184008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.184726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.185061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.185343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.186040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.188049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.199167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:48.231896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 889/1801] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:06:48 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:48.236956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69da84-a552-42ed-bd03-58e793243622 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.249179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69da84-a552-42ed-bd03-58e793243622 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.314912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:48.326274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.327089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.327359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.328049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.328689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.331168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.354214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 913/1802] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:06:48 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:06:48.358326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.369037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81f9aa-40ec-49e6-b3d3-a2b71158d06d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.373620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.374261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.374538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.374811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.375446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.377037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.387305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 890/1803] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:06:48.392044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.402411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9baec2-46bd-4df5-a991-421475f496a1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.407349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.408024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.408304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.408574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] PATH_INFO: `/v3/projects/f7522013907a4bb7bbd656c41b670569/users/3eccc1df3b384293b108b8fc78271d4c/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.424748 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7522013907a4bb7bbd656c41b670569, user_id=3eccc1df3b384293b108b8fc78271d4c, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.427957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.444075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 914/1804] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:48 2022] PUT /identity/v3/projects/f7522013907a4bb7bbd656c41b670569/users/3eccc1df3b384293b108b8fc78271d4c/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:48.448350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.458816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58c2f6-4697-4c34-a615-0f44a4869e12 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.521720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fae45d-cb19-4899-a742-167d67b87788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.522360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae45d-cb19-4899-a742-167d67b87788 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.522670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae45d-cb19-4899-a742-167d67b87788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.522942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae45d-cb19-4899-a742-167d67b87788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.523575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fae45d-cb19-4899-a742-167d67b87788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.525194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fae45d-cb19-4899-a742-167d67b87788 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.539594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 891/1805] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:06:48.541053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.552550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe8868-ed96-4fd3-be03-fc1880c41a27 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.557715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.558375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.558651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.558921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] PATH_INFO: `/v3/projects/f7522013907a4bb7bbd656c41b670569/users/3eccc1df3b384293b108b8fc78271d4c/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.560843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7522013907a4bb7bbd656c41b670569, user_id=3eccc1df3b384293b108b8fc78271d4c, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.563570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.576468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 915/1806] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:06:48 2022] PUT /identity/v3/projects/f7522013907a4bb7bbd656c41b670569/users/3eccc1df3b384293b108b8fc78271d4c/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:06:48.588999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.589341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.589815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.623835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] MFA Rules not processed for user `3eccc1df3b384293b108b8fc78271d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:48.625006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:48.704979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 892/1807] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:48.718047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.719772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:48.780222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1668084-e784-4067-a462-140a7945d085 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.784449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.785047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.785345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.785616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.795728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.797172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.807941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 916/1808] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:48.852595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:48.854058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:48.866836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df488cd-072d-441f-91bd-06b30c84a92a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:48.870574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2137f-2d1c-4d83-912b-15c9b8cdc940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:48.871180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2137f-2d1c-4d83-912b-15c9b8cdc940 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.871456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2137f-2d1c-4d83-912b-15c9b8cdc940 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.871720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2137f-2d1c-4d83-912b-15c9b8cdc940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.935822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2137f-2d1c-4d83-912b-15c9b8cdc940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:48.937266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2137f-2d1c-4d83-912b-15c9b8cdc940 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:48.941453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:48.950074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 893/1809] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:58.802724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=47,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:58.950636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:58.950906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:58.952745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.028891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59da4-944b-443e-9385-1967b03fff8a service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:59.033905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:59.034596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.034873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.035145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] PATH_INFO: `/v3/users/de1322b26ff94e2399716ff38d78c476` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.036290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1322b26ff94e2399716ff38d78c476)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:59.038372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:59.090723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1322b26ff94e2399716ff38d78c476'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.103575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1322b26ff94e2399716ff38d78c476'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.111694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1322b26ff94e2399716ff38d78c476'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.112428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1322b26ff94e2399716ff38d78c476'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.119571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1322b26ff94e2399716ff38d78c476'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.126104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 917/1810] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:58 2022] DELETE /identity/v3/users/de1322b26ff94e2399716ff38d78c476 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:59.131419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.131710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.132057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.153880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:59.154899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.232963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 894/1811] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:59.243053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:59.244530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.316413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5a09e-e2a2-48c1-bd61-303cf3c02b53 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:59.320303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a006e3-9902-4bd7-9b78-c9e4aad12829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:59.320913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a006e3-9902-4bd7-9b78-c9e4aad12829 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.321189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a006e3-9902-4bd7-9b78-c9e4aad12829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.321448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a006e3-9902-4bd7-9b78-c9e4aad12829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.331430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a006e3-9902-4bd7-9b78-c9e4aad12829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:59.332876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a006e3-9902-4bd7-9b78-c9e4aad12829 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:59.342739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 918/1812] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:06:59.429049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:59.430591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.443189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbbd21-de0d-4544-b811-8d063e108f1d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:59.447897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:59.448534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.448812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.449079 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] PATH_INFO: `/v3/users/1519c5416bfa4443827d0295d0e6e706` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.450216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1519c5416bfa4443827d0295d0e6e706)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:59.452137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:59.519904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1519c5416bfa4443827d0295d0e6e706'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.529383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1519c5416bfa4443827d0295d0e6e706'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.582382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1519c5416bfa4443827d0295d0e6e706'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.582382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1519c5416bfa4443827d0295d0e6e706'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.582382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1519c5416bfa4443827d0295d0e6e706'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.595034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 895/1813] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:59 2022] DELETE /identity/v3/users/1519c5416bfa4443827d0295d0e6e706 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:06:59.598220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.598593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.598789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.630137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:59.631471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.657874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:59.693621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:59.695170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.732631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 919/1814] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:59.742893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:59.744488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.804026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80332e6b-8a30-489d-a87d-261cb1d56c7d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:59.809804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:59.810629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.811236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.811373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] PATH_INFO: `/v3/projects/673fd028eadd4bc380ec8a50e2b3465f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.813885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673fd028eadd4bc380ec8a50e2b3465f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:59.814715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:59.836010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:59.838057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5814ea3-90c1-4ef5-89c8-7a200e868411 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:59.845469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:59.846747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.847292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.847573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.860819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:59.862377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:59.875089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 920/1815] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:59.876104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:59.879126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.886696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673fd028eadd4bc380ec8a50e2b3465f', 'request_id': 'req-80150a2a-f4ac-4b23-9a05-98c42a498676'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.892213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57955963-2502-42cc-b189-8b8d0f9aeb31 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:59.894230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673fd028eadd4bc380ec8a50e2b3465f', 'request_id': 'req-80150a2a-f4ac-4b23-9a05-98c42a498676'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.901558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673fd028eadd4bc380ec8a50e2b3465f', 'request_id': 'req-80150a2a-f4ac-4b23-9a05-98c42a498676'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.920123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] The token cache is being invalidate because project 673fd028eadd4bc380ec8a50e2b3465f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:06:59.920673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.922049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:06:59.924145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 896/1816] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:06:59 2022] DELETE /identity/v3/projects/673fd028eadd4bc380ec8a50e2b3465f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:06:59.928365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:59.929777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:59.969615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:59.970287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:59.970565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:59.970874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] PATH_INFO: `/v3/users/b56c77cb57d54036a60fdff39959098e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:59.976670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56c77cb57d54036a60fdff39959098e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:59.978792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:00.035563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80150a2a-f4ac-4b23-9a05-98c42a498676 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:00.041521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:00.042336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.042673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.042853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56c77cb57d54036a60fdff39959098e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.043075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] PATH_INFO: `/v3/users/2ca72ace2a174a9dba959f8139251958` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.044570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca72ace2a174a9dba959f8139251958)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:00.046724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:00.061579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56c77cb57d54036a60fdff39959098e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.064463 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:00.074516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56c77cb57d54036a60fdff39959098e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.075763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56c77cb57d54036a60fdff39959098e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.091807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2e52b07e-e1ee-4a51-80a2-521bea52a304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56c77cb57d54036a60fdff39959098e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.109625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 921/1817] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:59 2022] DELETE /identity/v3/users/b56c77cb57d54036a60fdff39959098e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:00.112127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.112643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.112948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.124780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca72ace2a174a9dba959f8139251958'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.139498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca72ace2a174a9dba959f8139251958'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.140338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:00.142087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:00.159043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca72ace2a174a9dba959f8139251958'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.160796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca72ace2a174a9dba959f8139251958'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.168840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca72ace2a174a9dba959f8139251958'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.177413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 897/1818] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:06:59 2022] DELETE /identity/v3/users/2ca72ace2a174a9dba959f8139251958 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:00.224250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 922/1819] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:00.238696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:00.241870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:00.272307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:00.316689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d22490-e3f9-4975-8219-17fdf731bad3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:00.320746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:00.321460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.321821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.322202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.405151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:00.406647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:00.418247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 898/1820] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:00.564368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:00.566860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:00.582700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b6f42-d307-4086-83f7-5c379c6ee729 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:00.589218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:00.590148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.590527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.590918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] PATH_INFO: `/v3/projects/02a1c9a43be54723a335ecfcc7bb2b77` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.592993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a1c9a43be54723a335ecfcc7bb2b77)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:00.595087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:00.667813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a1c9a43be54723a335ecfcc7bb2b77', 'request_id': 'req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.675758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a1c9a43be54723a335ecfcc7bb2b77', 'request_id': 'req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.687623 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a1c9a43be54723a335ecfcc7bb2b77', 'request_id': 'req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.727273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] The token cache is being invalidate because project 02a1c9a43be54723a335ecfcc7bb2b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:00.727589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.728414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.738412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 923/1821] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:00 2022] DELETE /identity/v3/projects/02a1c9a43be54723a335ecfcc7bb2b77 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:00.742140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:00.758914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3668d-2086-4ab9-8e8e-46d7292efd87 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:00.827120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:00.829019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:00.876317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:00.876978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.877150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.877799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] PATH_INFO: `/v3/users/5a2d5a370e3d480bbecf81a0378b2319` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.879252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2d5a370e3d480bbecf81a0378b2319)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:00.882076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:00.929432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35504d74-9e96-44df-a7d7-3b6f6b5c3bb5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:00.939529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:00.940535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.940820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.941878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] PATH_INFO: `/v3/projects/2ccbf25b6bbc4116b461e5c49c69bda1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.943427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccbf25b6bbc4116b461e5c49c69bda1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:00.947937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:00.957175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2d5a370e3d480bbecf81a0378b2319'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.973048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:00.973220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2d5a370e3d480bbecf81a0378b2319'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.993966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2d5a370e3d480bbecf81a0378b2319'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.994936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2d5a370e3d480bbecf81a0378b2319'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:00.997489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:01.005420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2d5a370e3d480bbecf81a0378b2319'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.017262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 899/1822] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:00 2022] DELETE /identity/v3/users/5a2d5a370e3d480bbecf81a0378b2319 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:01.018411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.020084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.040684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccbf25b6bbc4116b461e5c49c69bda1', 'request_id': 'req-aea35c6b-291c-4b2a-a56d-557bda97f1a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.047869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccbf25b6bbc4116b461e5c49c69bda1', 'request_id': 'req-aea35c6b-291c-4b2a-a56d-557bda97f1a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.058647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccbf25b6bbc4116b461e5c49c69bda1', 'request_id': 'req-aea35c6b-291c-4b2a-a56d-557bda97f1a2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.072582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] The token cache is being invalidate because project 2ccbf25b6bbc4116b461e5c49c69bda1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:01.072866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.074087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.076636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 924/1823] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:00 2022] DELETE /identity/v3/projects/2ccbf25b6bbc4116b461e5c49c69bda1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:01.080574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.082929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.097500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922340f-c946-4e5b-b6e0-55902d6a3baf admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.121799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.122433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.122716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.122988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] PATH_INFO: `/v3/projects/0e9c73a5accc4310b163fee52fdb9251` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.124187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9c73a5accc4310b163fee52fdb9251)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.126724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.165145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea35c6b-291c-4b2a-a56d-557bda97f1a2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.183364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9c73a5accc4310b163fee52fdb9251', 'request_id': 'req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.190790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9c73a5accc4310b163fee52fdb9251', 'request_id': 'req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.198409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9c73a5accc4310b163fee52fdb9251', 'request_id': 'req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.218071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] The token cache is being invalidate because project 0e9c73a5accc4310b163fee52fdb9251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:01.218373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.219041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4df704c-5c00-4ab4-86d2-67bdf3ec930d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.220947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 900/1824] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:01 2022] DELETE /identity/v3/projects/0e9c73a5accc4310b163fee52fdb9251 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:01.225955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229b99c-50ab-4771-bf7a-ec76aa03bd1f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.226355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229b99c-50ab-4771-bf7a-ec76aa03bd1f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.226703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229b99c-50ab-4771-bf7a-ec76aa03bd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.239561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.240452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.240832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.241225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] PATH_INFO: `/v3/users/dc30a2ab5003454c8b0171026aecb091` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.242601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc30a2ab5003454c8b0171026aecb091)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.244953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.254609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8229b99c-50ab-4771-bf7a-ec76aa03bd1f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:01.256099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8229b99c-50ab-4771-bf7a-ec76aa03bd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.268593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 901/1825] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:01.277464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68852-bf79-41b7-bc27-e921e1c383b1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.277934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68852-bf79-41b7-bc27-e921e1c383b1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.278359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68852-bf79-41b7-bc27-e921e1c383b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.305923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-54f68852-bf79-41b7-bc27-e921e1c383b1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:01.307757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-54f68852-bf79-41b7-bc27-e921e1c383b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.309263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc30a2ab5003454c8b0171026aecb091'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.311436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:01.327685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc30a2ab5003454c8b0171026aecb091'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.335125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 902/1826] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:01.338546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc30a2ab5003454c8b0171026aecb091'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.339475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc30a2ab5003454c8b0171026aecb091'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.339773 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68852-bf79-41b7-bc27-e921e1c383b1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.351013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68852-bf79-41b7-bc27-e921e1c383b1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.358969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc30a2ab5003454c8b0171026aecb091'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.363542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.364297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.364671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.366465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.367232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:01.367983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.370893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.377605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 925/1827] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:01 2022] DELETE /identity/v3/users/dc30a2ab5003454c8b0171026aecb091 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:01.391457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 903/1828] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:01.400941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.400941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.482258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21383-deef-4514-8f90-ccfbadcae256 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.487623 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.488280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.488571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.488846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.489493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.491472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.521756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.523740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.536192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 926/1829] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:01 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:01.539812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.544796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff029dfa-7a87-4734-8362-0e07f22bb73f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.626242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:01.637226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b85f5-79b2-4012-9464-12536a10234d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.638215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.638876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.639633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.640223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] PATH_INFO: `/v3/projects/db88b32425314286972645f72db7aec7` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.643076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.643222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db88b32425314286972645f72db7aec7)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.643998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.644341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.644608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.645329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.646228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.647442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.679419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 927/1830] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:01 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:01.682587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.695758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06524101-1a3d-4e2a-83cc-e174609cc12a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.713589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db88b32425314286972645f72db7aec7', 'request_id': 'req-5a45f009-9f32-47d4-8789-44230e0e4b39'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.720226 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db88b32425314286972645f72db7aec7', 'request_id': 'req-5a45f009-9f32-47d4-8789-44230e0e4b39'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.732852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db88b32425314286972645f72db7aec7', 'request_id': 'req-5a45f009-9f32-47d4-8789-44230e0e4b39'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.744549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:01.757005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] The token cache is being invalidate because project db88b32425314286972645f72db7aec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:01.757389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.758481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.761307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 904/1831] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:01 2022] DELETE /identity/v3/projects/db88b32425314286972645f72db7aec7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:01.764511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.766932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.772349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.773041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.774469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.774760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.775420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.777736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.786131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f009-9f32-47d4-8789-44230e0e4b39 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.793354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.793354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.793354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.793354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] PATH_INFO: `/v3/users/3eccc1df3b384293b108b8fc78271d4c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.794680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eccc1df3b384293b108b8fc78271d4c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.797159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.797420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 928/1832] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:01.798638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.800969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.816364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b1cb-87fb-42ad-88eb-fe8825499d75 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.823666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.824282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.824561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.824850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.825479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.827135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.853780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 929/1833] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:01.859301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.872394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eccc1df3b384293b108b8fc78271d4c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.877014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cb4ad-e4e6-4ae8-8c0e-3fbc46b1e1b0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.886312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9ac3c-8dd0-4d2a-8996-b679e82e5d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.887008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ac3c-8dd0-4d2a-8996-b679e82e5d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.887885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eccc1df3b384293b108b8fc78271d4c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.891488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ac3c-8dd0-4d2a-8996-b679e82e5d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.891488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ac3c-8dd0-4d2a-8996-b679e82e5d55 admin admin] PATH_INFO: `/v3/projects/5a72d8c195a64a04a30f14b25b75c156/users/702213ad57c8474eae52fc9314f357ce/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.915475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eccc1df3b384293b108b8fc78271d4c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.916305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eccc1df3b384293b108b8fc78271d4c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.924637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eccc1df3b384293b108b8fc78271d4c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:01.925596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9ac3c-8dd0-4d2a-8996-b679e82e5d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a72d8c195a64a04a30f14b25b75c156, user_id=702213ad57c8474eae52fc9314f357ce, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.929077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9ac3c-8dd0-4d2a-8996-b679e82e5d55 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.934919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 905/1834] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:01 2022] DELETE /identity/v3/users/3eccc1df3b384293b108b8fc78271d4c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:01.935917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.937628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.960099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 930/1835] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:01 2022] PUT /identity/v3/projects/5a72d8c195a64a04a30f14b25b75c156/users/702213ad57c8474eae52fc9314f357ce/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:01.971230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.971575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.971847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.006882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] MFA Rules not processed for user `702213ad57c8474eae52fc9314f357ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:02.008127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.047869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:02.055060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d068b5-0101-46fe-9e57-3c66e8bcccfd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.086048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 931/1836] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:02.090640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.092458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.176147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.178387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.179902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.180379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] PATH_INFO: `/v3/projects/a65c24e216724bf5960cc427960e883c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.181973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65c24e216724bf5960cc427960e883c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.185156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.197766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174e9ca-d795-40af-ac72-ae73a2cbbd7a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.211178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.211444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.211733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.212034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.212655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.219410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.264507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65c24e216724bf5960cc427960e883c', 'request_id': 'req-a01abb35-e427-47a9-85a1-7776ed04220f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.272242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 932/1837] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:02 2022] POST /identity/v3/projects => generated 387 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:02.280536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65c24e216724bf5960cc427960e883c', 'request_id': 'req-a01abb35-e427-47a9-85a1-7776ed04220f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.287150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.292507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65c24e216724bf5960cc427960e883c', 'request_id': 'req-a01abb35-e427-47a9-85a1-7776ed04220f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.334739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] The token cache is being invalidate because project a65c24e216724bf5960cc427960e883c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:02.336164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:02.338212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.340928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a01abb35-e427-47a9-85a1-7776ed04220f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.344950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 906/1838] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:01 2022] DELETE /identity/v3/projects/a65c24e216724bf5960cc427960e883c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:02.350588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54742e03-ff3e-4392-af4a-c1c702e121c9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.350949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54742e03-ff3e-4392-af4a-c1c702e121c9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.351841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54742e03-ff3e-4392-af4a-c1c702e121c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.386100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-54742e03-ff3e-4392-af4a-c1c702e121c9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:02.387532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-54742e03-ff3e-4392-af4a-c1c702e121c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.442483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23caa-7bfd-441b-a778-9bad402534ba admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.448544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.449203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.449644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.449921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.450566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.453223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.474809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 907/1839] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:02.484147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34d6a-e6d9-4675-9e0b-fb6468bc932c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.484713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34d6a-e6d9-4675-9e0b-fb6468bc932c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.485172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb34d6a-e6d9-4675-9e0b-fb6468bc932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.488768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 933/1840] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:02 2022] POST /identity/v3/users => generated 300 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:02.490563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.494599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.509395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c40493-fafe-482a-9717-4042b9092281 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.512884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cdb34d6a-e6d9-4675-9e0b-fb6468bc932c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:02.514805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cdb34d6a-e6d9-4675-9e0b-fb6468bc932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.515308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.516024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.517425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.517425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.517975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.520073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.528732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 908/1841] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:02.530044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb34d6a-e6d9-4675-9e0b-fb6468bc932c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.536250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 934/1842] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:02.540456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.542490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb34d6a-e6d9-4675-9e0b-fb6468bc932c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.548012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.548735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.549065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.549389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.550183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:02.550574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.552472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.557001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ec636-49e3-48ad-86b7-f3697732a567 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.565097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.565929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.566218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.566496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] PATH_INFO: `/v3/projects/dbc8155e0eb747f2a36014a7f85b2464/users/5f4b26cc04544f058bff7251101e47b7/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.575797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 909/1843] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:02.576731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.578536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.588915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8155e0eb747f2a36014a7f85b2464, user_id=5f4b26cc04544f058bff7251101e47b7, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.593652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.593817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ccf35-6ca0-4f41-a04b-c75f7dd0880b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.599798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.600450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.600773 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.601048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] PATH_INFO: `/v3/projects/f7522013907a4bb7bbd656c41b670569` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.602206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7522013907a4bb7bbd656c41b670569)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.604184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.618172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 935/1844] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:02 2022] PUT /identity/v3/projects/dbc8155e0eb747f2a36014a7f85b2464/users/5f4b26cc04544f058bff7251101e47b7/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:02.619148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.619429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:02.707594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f447-6530-41d7-94b6-ca80ae018709 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.713734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7522013907a4bb7bbd656c41b670569', 'request_id': 'req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.757475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7522013907a4bb7bbd656c41b670569', 'request_id': 'req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.803134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7522013907a4bb7bbd656c41b670569', 'request_id': 'req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.824605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] The token cache is being invalidate because project f7522013907a4bb7bbd656c41b670569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:02.824940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.830191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:02.831576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cfa3c-1625-49d4-892f-1cefc39cbc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.832498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cfa3c-1625-49d4-892f-1cefc39cbc18 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.833272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 910/1845] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:02 2022] DELETE /identity/v3/projects/f7522013907a4bb7bbd656c41b670569 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:02.833609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cfa3c-1625-49d4-892f-1cefc39cbc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.834097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cfa3c-1625-49d4-892f-1cefc39cbc18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.834475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.835112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cfa3c-1625-49d4-892f-1cefc39cbc18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.836705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.837749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cfa3c-1625-49d4-892f-1cefc39cbc18 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.864286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bc3c6-bed2-4a10-9f7a-f88fd4a57cc9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.877732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 936/1846] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:02 2022] POST /identity/v3/projects => generated 363 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:02.878825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:02.879526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.879871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.880223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.880619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db2320-da3b-4b0b-a03c-3603795259af None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:02.880989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:02.881753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db2320-da3b-4b0b-a03c-3603795259af None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:02.882214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db2320-da3b-4b0b-a03c-3603795259af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:02.882866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:02.895971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:02.908409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 911/1847] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:02.909140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:02.912965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:02.931405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28aef2-709a-417d-b036-3ccf4832732c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:02.932717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-31db2320-da3b-4b0b-a03c-3603795259af None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:02.934077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31db2320-da3b-4b0b-a03c-3603795259af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:03.026791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.027591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.028613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.028613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.029294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.031580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.070053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 937/1848] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:03.072479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 912/1849] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:02 2022] POST /identity/v3/users => generated 282 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:03.073697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.079373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72ed9-0021-4c6b-9fca-a9927717c738 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.079800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72ed9-0021-4c6b-9fca-a9927717c738 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.080193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72ed9-0021-4c6b-9fca-a9927717c738 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.082049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 938/1850] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:07:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:03.087914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72ed9-0021-4c6b-9fca-a9927717c738 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.094393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf3a60-17b8-439a-93fa-25cb22337040 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.111734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505134c2-7b66-4c3d-a86c-fe1cfe04ea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.113471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505134c2-7b66-4c3d-a86c-fe1cfe04ea61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.114246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505134c2-7b66-4c3d-a86c-fe1cfe04ea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.114246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505134c2-7b66-4c3d-a86c-fe1cfe04ea61 admin admin] PATH_INFO: `/v3/projects/dbc8155e0eb747f2a36014a7f85b2464/users/5f4b26cc04544f058bff7251101e47b7/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.116526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505134c2-7b66-4c3d-a86c-fe1cfe04ea61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc8155e0eb747f2a36014a7f85b2464, user_id=5f4b26cc04544f058bff7251101e47b7, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.120531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505134c2-7b66-4c3d-a86c-fe1cfe04ea61 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.137591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 913/1851] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:03 2022] PUT /identity/v3/projects/dbc8155e0eb747f2a36014a7f85b2464/users/5f4b26cc04544f058bff7251101e47b7/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:03.139182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd24cdf-55fc-41a1-9ace-b97fce953d5f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.139766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd24cdf-55fc-41a1-9ace-b97fce953d5f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.140374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd24cdf-55fc-41a1-9ace-b97fce953d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.160787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72ed9-0021-4c6b-9fca-a9927717c738 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.192194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fbd24cdf-55fc-41a1-9ace-b97fce953d5f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:03.193638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fbd24cdf-55fc-41a1-9ace-b97fce953d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:03.276631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 914/1852] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:03.281338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a603e5-6404-4e86-8e3f-db99313f08b7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.281635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a603e5-6404-4e86-8e3f-db99313f08b7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.281937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a603e5-6404-4e86-8e3f-db99313f08b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.291251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:03.304921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d0a603e5-6404-4e86-8e3f-db99313f08b7 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:03.306175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d0a603e5-6404-4e86-8e3f-db99313f08b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:03.386581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.387254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.387548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.387824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.390312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.392585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.398334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 915/1853] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:07:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:03.399892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcdaf-a5be-4260-a92a-d34f24bde589 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.400322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcdaf-a5be-4260-a92a-d34f24bde589 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.400780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcdaf-a5be-4260-a92a-d34f24bde589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.434332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 939/1854] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:03.434332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.445371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e21c-06fd-4fad-bba5-0d7922332441 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.450105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a3ffcdaf-a5be-4260-a92a-d34f24bde589 None None] MFA Rules not processed for user `5f4b26cc04544f058bff7251101e47b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:03.451730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a3ffcdaf-a5be-4260-a92a-d34f24bde589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:03.454690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.457163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.458244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.459315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.460620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:03.461178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.463873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.480582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 940/1855] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:03.484856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.486816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:03.505809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461e56c-fb47-4019-b405-7d4209e7646f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.512080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.512849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.513234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.513616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.524175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.525990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.537450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 941/1856] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:03.538193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 916/1857] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:03.546840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.548917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcdaf-a5be-4260-a92a-d34f24bde589 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.562164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcdaf-a5be-4260-a92a-d34f24bde589 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.562929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff77f-19c3-415c-b69c-5f64a000e7e5 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.567993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.568408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.569236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.569336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.569423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.569643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.569804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.570078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.570459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.570856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.572691 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.574387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.600674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 942/1858] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:03.601759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.603697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:03.611860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 917/1859] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:03 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:03.612947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.671654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761ae6b-20ce-4b3d-9ca9-355690ea33d3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.684000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a1ef-970b-4692-8095-f33bea1ec4f9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.684582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:03.689730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.690261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.690655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.690929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] PATH_INFO: `/v3/projects/d420a78c7f7142fba1bbc7bdd4233241/users/1b5c80a66b704fb88b6d723f45fc324b/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.693508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:03.711045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d420a78c7f7142fba1bbc7bdd4233241, user_id=1b5c80a66b704fb88b6d723f45fc324b, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.716211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.729764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7786a21-0dca-4f79-b964-c8eea2d486e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.730641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786a21-0dca-4f79-b964-c8eea2d486e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.731054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786a21-0dca-4f79-b964-c8eea2d486e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.731449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786a21-0dca-4f79-b964-c8eea2d486e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.733366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 918/1860] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:03 2022] PUT /identity/v3/projects/d420a78c7f7142fba1bbc7bdd4233241/users/1b5c80a66b704fb88b6d723f45fc324b/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:03.734234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.809941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7786a21-0dca-4f79-b964-c8eea2d486e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.810214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c533d94-410f-448c-9ca8-4de25229ee6d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.811531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7786a21-0dca-4f79-b964-c8eea2d486e0 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.816925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.818022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.818301 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.818575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.819212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.821172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.824619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 943/1861] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:03.825909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7569e2d-f295-4141-93fa-363ae3c7ecd7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.826192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7569e2d-f295-4141-93fa-363ae3c7ecd7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.826469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7569e2d-f295-4141-93fa-363ae3c7ecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.849909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 919/1862] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:03 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:03.857877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.864350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a7569e2d-f295-4141-93fa-363ae3c7ecd7 None None] MFA Rules not processed for user `1b5c80a66b704fb88b6d723f45fc324b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:03.865858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7569e2d-f295-4141-93fa-363ae3c7ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:03.872402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a107ede-723f-4a82-a1c8-b737af4c407f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.878076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.878686 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.878969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.879291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.879875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.881518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.892949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 920/1863] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:07:03.897076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.910028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e949-ff32-4c71-838e-4b1028e6497b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.915630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.916318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.916680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.916985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.917835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.919439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.940413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 921/1864] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:07:03.945055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.947404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 944/1865] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:03.951552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7569e2d-f295-4141-93fa-363ae3c7ecd7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:03.957132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729e5f8-b8f5-4052-867a-c7d1d009f86b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.962610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7569e2d-f295-4141-93fa-363ae3c7ecd7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:03.963868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.964551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.964834 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.965107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] PATH_INFO: `/v3/projects/8b8cab8b809c449a9699686cbcb59400/users/5bc5b816fb9f4d858c656865415b5d7f/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.968068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7641de5-09f8-4335-ab67-e3ff1bc01819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:03.968739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7641de5-09f8-4335-ab67-e3ff1bc01819 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:03.969110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7641de5-09f8-4335-ab67-e3ff1bc01819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:03.969364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7641de5-09f8-4335-ab67-e3ff1bc01819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:03.970091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7641de5-09f8-4335-ab67-e3ff1bc01819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.972291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7641de5-09f8-4335-ab67-e3ff1bc01819 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:03.983230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8cab8b809c449a9699686cbcb59400, user_id=5bc5b816fb9f4d858c656865415b5d7f, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:03.986218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.002793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 922/1866] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:03 2022] PUT /identity/v3/projects/8b8cab8b809c449a9699686cbcb59400/users/5bc5b816fb9f4d858c656865415b5d7f/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:04.005609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 945/1867] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:03 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:04.009954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.015150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a863d69-d524-440f-885a-b368b51deea2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.015538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a863d69-d524-440f-885a-b368b51deea2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.015942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a863d69-d524-440f-885a-b368b51deea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.048292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6a863d69-d524-440f-885a-b368b51deea2 None None] MFA Rules not processed for user `5bc5b816fb9f4d858c656865415b5d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:04.049470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6a863d69-d524-440f-885a-b368b51deea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:04.081126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:04.083831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b55c8-5cd4-4df4-9452-74a5aee4be81 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.121527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:04.137891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 946/1868] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:04.142663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a863d69-d524-440f-885a-b368b51deea2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.169791 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.170768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.171244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.171710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.172797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.177499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.214430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 923/1869] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:04 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:04.218368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.229094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8653376-41ba-4a1a-b8b4-136d823051a2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.235810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.236560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.236846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.237211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.237902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.239507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.244284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a863d69-d524-440f-885a-b368b51deea2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.250232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.250745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 924/1870] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:04.251525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.252028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.252521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.253480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.254987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.256038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.267025 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d2f2d-58b0-4198-bad5-fecae25347b8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.272278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.272957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.273290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.273570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] PATH_INFO: `/v3/projects/d010ada004d44207b7e168f345ecbc8a/users/db446615ec984177b03aaf0d949f2c06/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.291649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d010ada004d44207b7e168f345ecbc8a, user_id=db446615ec984177b03aaf0d949f2c06, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.294855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.303308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 947/1871] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:04 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:04.307688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.314198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 925/1872] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:04 2022] PUT /identity/v3/projects/d010ada004d44207b7e168f345ecbc8a/users/db446615ec984177b03aaf0d949f2c06/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:04.319530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.426215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2ba36-4272-4019-903f-3dd41e9845e0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.432309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.433073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.433452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.433860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.434619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.436649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.439893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd38bb-53bb-468e-8b61-a094bc8ee8f0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.448323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.448750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.449127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.449495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.450327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.452625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.453862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 926/1873] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:04.454793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.466510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734ec5b-a138-4052-ac67-e42faff27507 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.475522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d8dd3-7261-4b3f-b72c-98d9732155cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.476260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d8dd3-7261-4b3f-b72c-98d9732155cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.476542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d8dd3-7261-4b3f-b72c-98d9732155cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.476866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d8dd3-7261-4b3f-b72c-98d9732155cc admin admin] PATH_INFO: `/v3/projects/d010ada004d44207b7e168f345ecbc8a/users/db446615ec984177b03aaf0d949f2c06/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.478887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d8dd3-7261-4b3f-b72c-98d9732155cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d010ada004d44207b7e168f345ecbc8a, user_id=db446615ec984177b03aaf0d949f2c06, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.481718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 948/1874] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:04 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:04.481873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d8dd3-7261-4b3f-b72c-98d9732155cc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.484573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.516696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 927/1875] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:04 2022] PUT /identity/v3/projects/d010ada004d44207b7e168f345ecbc8a/users/db446615ec984177b03aaf0d949f2c06/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:04.530894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c2fe8-c890-47e7-95e9-1c512dd57d49 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.531310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c2fe8-c890-47e7-95e9-1c512dd57d49 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.531713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c2fe8-c890-47e7-95e9-1c512dd57d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.542344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170ff96-40ef-450c-abc7-0f8e05709c9c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.593928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-066c2fe8-c890-47e7-95e9-1c512dd57d49 None None] MFA Rules not processed for user `db446615ec984177b03aaf0d949f2c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:04.596595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-066c2fe8-c890-47e7-95e9-1c512dd57d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:04.652658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:04.710597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.711247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.711521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.711801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.712456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.716730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.743696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 949/1876] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:04.745267 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.767397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:04.768199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b272b6-9a58-4b93-ba65-c60de2afd844 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.775405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.776873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.777505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.777976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] PATH_INFO: `/v3/users/702213ad57c8474eae52fc9314f357ce` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.779622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=702213ad57c8474eae52fc9314f357ce)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.782155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.791116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 928/1877] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:04.792545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c2fe8-c890-47e7-95e9-1c512dd57d49 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.806984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c2fe8-c890-47e7-95e9-1c512dd57d49 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:04.812916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:04.813896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:04.814181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:04.814451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] PATH_INFO: `/v3/projects/f4dce6088eb042cc96db922402fde842/users/0469f6e6d1ad4daeaa082b198b6cfb17/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:04.834371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4dce6088eb042cc96db922402fde842, user_id=0469f6e6d1ad4daeaa082b198b6cfb17, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:04.839048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:04.861082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 929/1878] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:04 2022] PUT /identity/v3/projects/f4dce6088eb042cc96db922402fde842/users/0469f6e6d1ad4daeaa082b198b6cfb17/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:04.862001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.865874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:04.873371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702213ad57c8474eae52fc9314f357ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:04.896099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702213ad57c8474eae52fc9314f357ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:04.911315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702213ad57c8474eae52fc9314f357ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:04.911988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702213ad57c8474eae52fc9314f357ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:04.920480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702213ad57c8474eae52fc9314f357ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:04.929197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 950/1879] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:04 2022] DELETE /identity/v3/users/702213ad57c8474eae52fc9314f357ce => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:04.932337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:04.934421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:04.956562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621cc1e-3634-456c-ad55-616e8ea502d9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:05.011638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336cf55-57a3-47d1-8baf-f129307378fd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:05.017332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f8ae-c561-48bc-85a9-c4314664bb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:05.018007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f8ae-c561-48bc-85a9-c4314664bb17 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.018309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f8ae-c561-48bc-85a9-c4314664bb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.018853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f8ae-c561-48bc-85a9-c4314664bb17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.019655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69f8ae-c561-48bc-85a9-c4314664bb17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:05.021300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69f8ae-c561-48bc-85a9-c4314664bb17 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:05.027498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88cc67-d819-495b-a915-5c9fcc9ac7fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:05.028210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88cc67-d819-495b-a915-5c9fcc9ac7fa service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.028565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88cc67-d819-495b-a915-5c9fcc9ac7fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.029066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88cc67-d819-495b-a915-5c9fcc9ac7fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.030904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cb88cc67-d819-495b-a915-5c9fcc9ac7fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:05.036818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 951/1880] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:05.039278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95777cc6-c14d-4569-aa3f-80cea018068d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.039660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95777cc6-c14d-4569-aa3f-80cea018068d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.039923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95777cc6-c14d-4569-aa3f-80cea018068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.062889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-95777cc6-c14d-4569-aa3f-80cea018068d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:05.064830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95777cc6-c14d-4569-aa3f-80cea018068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:05.073102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 952/1881] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:05.077095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95777cc6-c14d-4569-aa3f-80cea018068d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:05.087912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95777cc6-c14d-4569-aa3f-80cea018068d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:05.093474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:05.093616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.094060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.094300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] PATH_INFO: `/v3/projects/f4dce6088eb042cc96db922402fde842/users/0469f6e6d1ad4daeaa082b198b6cfb17/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.096268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4dce6088eb042cc96db922402fde842, user_id=0469f6e6d1ad4daeaa082b198b6cfb17, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:05.099136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:05.114094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 953/1882] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:05 2022] PUT /identity/v3/projects/f4dce6088eb042cc96db922402fde842/users/0469f6e6d1ad4daeaa082b198b6cfb17/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:05.114094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:05.115182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:05.126010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88cc67-d819-495b-a915-5c9fcc9ac7fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:05.127690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88cc67-d819-495b-a915-5c9fcc9ac7fa service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:05.138545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 930/1883] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:05.140709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.141016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.141607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.147858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:05.184050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] MFA Rules not processed for user `0469f6e6d1ad4daeaa082b198b6cfb17`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:05.184050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:05.186331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4575a20-b60b-4f3d-95a8-70364132ad9c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:05.191146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:05.191859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.192204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.193151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.212456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:05.265596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:05.268859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:05.279128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 931/1884] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:05.285377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 954/1885] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:05.292149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:05.293919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:05.381933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767b36-3919-4364-9f14-8004f2bdd475 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:05.387671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:05.388533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.388892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.389281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.401013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:05.403410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:05.415247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 932/1886] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:05.562500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:05.576723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8137-799d-4a9a-a69d-f15f1fe9f468 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:05.588345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:05.588984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.589266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.589537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.603305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:05.735301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:05.738156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:05.760352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 955/1887] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:05.885918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:05.888334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:05.911416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef82d4f-4196-47e3-84d4-a5b08184b267 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:05.942415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:05.943086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.943461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.943627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] PATH_INFO: `/v3/users/1b5c80a66b704fb88b6d723f45fc324b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.946559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b5c80a66b704fb88b6d723f45fc324b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:05.948674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:05.986149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:05.988979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.017784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706878e-0368-4fc3-9228-ebcfe75348b4 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:06.027438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:06.028194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.028491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.028763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] PATH_INFO: `/v3/projects/5a72d8c195a64a04a30f14b25b75c156` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.031315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a72d8c195a64a04a30f14b25b75c156)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:06.035958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:06.055317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b5c80a66b704fb88b6d723f45fc324b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.065922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b5c80a66b704fb88b6d723f45fc324b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.099193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b5c80a66b704fb88b6d723f45fc324b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.099976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b5c80a66b704fb88b6d723f45fc324b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.108694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f1cbc199-c336-437b-ab40-9733d451d2b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b5c80a66b704fb88b6d723f45fc324b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.113904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a72d8c195a64a04a30f14b25b75c156', 'request_id': 'req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.131160 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a72d8c195a64a04a30f14b25b75c156', 'request_id': 'req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.131282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 933/1888] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:05 2022] DELETE /identity/v3/users/1b5c80a66b704fb88b6d723f45fc324b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:06.141763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a72d8c195a64a04a30f14b25b75c156', 'request_id': 'req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.142245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.142245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.142245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.142882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:06.152683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] The token cache is being invalidate because project 5a72d8c195a64a04a30f14b25b75c156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:06.153263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.156230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.158185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 956/1889] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:05 2022] DELETE /identity/v3/projects/5a72d8c195a64a04a30f14b25b75c156 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:06.163650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:06.165120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.173073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:06.174431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.213347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:06.243024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1796-7b9c-4763-a12d-b129fbf0ebf6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:06.244082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 934/1890] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:06.244776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:06.246152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.251692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:06.252338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.252594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.252902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] PATH_INFO: `/v3/users/5f4b26cc04544f058bff7251101e47b7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.254507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f4b26cc04544f058bff7251101e47b7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:06.256657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:06.258795 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebf82c-d5e2-4ae1-b586-9b83af88ec86 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:06.263985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:06.264576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.264816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.265209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] PATH_INFO: `/v3/users/5bc5b816fb9f4d858c656865415b5d7f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.266782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc5b816fb9f4d858c656865415b5d7f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:06.268780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:06.306243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f4b26cc04544f058bff7251101e47b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.317859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc5b816fb9f4d858c656865415b5d7f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.320031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f4b26cc04544f058bff7251101e47b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.326954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc5b816fb9f4d858c656865415b5d7f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.333198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f4b26cc04544f058bff7251101e47b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.333917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f4b26cc04544f058bff7251101e47b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.340148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc5b816fb9f4d858c656865415b5d7f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.340848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc5b816fb9f4d858c656865415b5d7f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.342300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f4b26cc04544f058bff7251101e47b7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.348018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42432d43-5fb3-4bf5-8a5b-3c2d20b77669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc5b816fb9f4d858c656865415b5d7f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.348893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 957/1891] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:06 2022] DELETE /identity/v3/users/5f4b26cc04544f058bff7251101e47b7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:06.349731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:06.351255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.357802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 935/1892] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:06 2022] DELETE /identity/v3/users/5bc5b816fb9f4d858c656865415b5d7f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:06.362211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.363405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.363688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.384485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:06.385906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.422722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f015690-badf-417c-a0dc-6c2303ec16d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:06.454685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 936/1893] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:06.463675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:06.466308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:06.468500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.489023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:06.489923 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.490263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.490606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.492532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.506518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:06.508128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:06.520421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 958/1894] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:06.542246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecadf7-8258-4fd4-990a-a8a34c2a3064 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:06.547366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:06.548182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.548599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.549157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.555964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:06.559702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:06.561166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:06.575272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 937/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:07:06.736311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:06.738229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.762537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea65f9b-25e9-40c2-9a7c-3865d21fa89f service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:06.774656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:06.775338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.775595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.775951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] PATH_INFO: `/v3/projects/dbc8155e0eb747f2a36014a7f85b2464` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.782580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc8155e0eb747f2a36014a7f85b2464)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:06.785401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:06.865399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc8155e0eb747f2a36014a7f85b2464', 'request_id': 'req-290cca91-2364-4628-aabe-e7265f8a59c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.872089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc8155e0eb747f2a36014a7f85b2464', 'request_id': 'req-290cca91-2364-4628-aabe-e7265f8a59c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.889980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc8155e0eb747f2a36014a7f85b2464', 'request_id': 'req-290cca91-2364-4628-aabe-e7265f8a59c1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.897059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:06.898741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.906136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] The token cache is being invalidate because project dbc8155e0eb747f2a36014a7f85b2464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:06.906505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.907217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-290cca91-2364-4628-aabe-e7265f8a59c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:06.912199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 959/1896] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:06 2022] DELETE /identity/v3/projects/dbc8155e0eb747f2a36014a7f85b2464 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:06.914467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bc8c0-612e-4ce1-bef5-49f9080a28b2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:06.914886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bc8c0-612e-4ce1-bef5-49f9080a28b2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:06.915180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bc8c0-612e-4ce1-bef5-49f9080a28b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.919423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca9efa-b74c-42f5-88fb-e93077c697ca service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:06.938227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bf7bc8c0-612e-4ce1-bef5-49f9080a28b2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:06.938227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bf7bc8c0-612e-4ce1-bef5-49f9080a28b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.039898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.040964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.041461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.042002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] PATH_INFO: `/v3/projects/d420a78c7f7142fba1bbc7bdd4233241` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.043402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d420a78c7f7142fba1bbc7bdd4233241)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.045500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 960/1897] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:07.046034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.048194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862d146-107d-49ec-a654-22ceff2052ee None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.048775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862d146-107d-49ec-a654-22ceff2052ee None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.049060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862d146-107d-49ec-a654-22ceff2052ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.074775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f862d146-107d-49ec-a654-22ceff2052ee None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:07.076582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f862d146-107d-49ec-a654-22ceff2052ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.092821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 961/1898] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:07.098285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862d146-107d-49ec-a654-22ceff2052ee None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.111361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862d146-107d-49ec-a654-22ceff2052ee None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.116147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd420a78c7f7142fba1bbc7bdd4233241', 'request_id': 'req-ce4eb272-969b-4fa7-9961-793356a1155b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.124682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd420a78c7f7142fba1bbc7bdd4233241', 'request_id': 'req-ce4eb272-969b-4fa7-9961-793356a1155b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.136444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd420a78c7f7142fba1bbc7bdd4233241', 'request_id': 'req-ce4eb272-969b-4fa7-9961-793356a1155b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.173222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] The token cache is being invalidate because project d420a78c7f7142fba1bbc7bdd4233241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:07.175065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.178923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.182701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:07.184278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 938/1899] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:06 2022] DELETE /identity/v3/projects/d420a78c7f7142fba1bbc7bdd4233241 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:07.185654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.192606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.199674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.200647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.201037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.201408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.202322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:07.202722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.204527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.224543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 962/1900] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:07.224543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.225812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.237525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4eb272-969b-4fa7-9961-793356a1155b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.251404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cb010-46dc-4d19-bc46-ceed604be1d4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.264698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.265467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.265779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.265899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.266105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] PATH_INFO: `/v3/users/db446615ec984177b03aaf0d949f2c06` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.266745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.267024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.267302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] PATH_INFO: `/v3/projects/8b8cab8b809c449a9699686cbcb59400` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.268146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db446615ec984177b03aaf0d949f2c06)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.268484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b8cab8b809c449a9699686cbcb59400)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.270467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.272044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.327526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b8cab8b809c449a9699686cbcb59400', 'request_id': 'req-f2e6822b-9a59-44e3-b279-9f1770670c48'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.328062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db446615ec984177b03aaf0d949f2c06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.334701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b8cab8b809c449a9699686cbcb59400', 'request_id': 'req-f2e6822b-9a59-44e3-b279-9f1770670c48'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.341771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b8cab8b809c449a9699686cbcb59400', 'request_id': 'req-f2e6822b-9a59-44e3-b279-9f1770670c48'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.345900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db446615ec984177b03aaf0d949f2c06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.354251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db446615ec984177b03aaf0d949f2c06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.355355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db446615ec984177b03aaf0d949f2c06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.356857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] The token cache is being invalidate because project 8b8cab8b809c449a9699686cbcb59400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:07.357739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.358471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.360871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 939/1901] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:07 2022] DELETE /identity/v3/projects/8b8cab8b809c449a9699686cbcb59400 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:07.361823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.363923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.367602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db446615ec984177b03aaf0d949f2c06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.374550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 963/1902] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:07 2022] DELETE /identity/v3/users/db446615ec984177b03aaf0d949f2c06 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:07.375385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.377179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.389044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:07.465275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6822b-9a59-44e3-b279-9f1770670c48 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.506220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b86c-5248-427f-a14f-ba97f96ba7e3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.512979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.513818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.514289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.514714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] PATH_INFO: `/v3/users/0469f6e6d1ad4daeaa082b198b6cfb17` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.516947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0469f6e6d1ad4daeaa082b198b6cfb17)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.519912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.542878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.544359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.544850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.545304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.546382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.548751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.589711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 940/1903] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:07 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:07.594123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.595827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0469f6e6d1ad4daeaa082b198b6cfb17'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.606522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:07.615605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0469f6e6d1ad4daeaa082b198b6cfb17'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.629016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0469f6e6d1ad4daeaa082b198b6cfb17'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.631817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0469f6e6d1ad4daeaa082b198b6cfb17'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.645565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0469f6e6d1ad4daeaa082b198b6cfb17'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.655287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 964/1904] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:07 2022] DELETE /identity/v3/users/0469f6e6d1ad4daeaa082b198b6cfb17 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:07.687872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549e06a-39a3-4bf7-aa95-988ba133794e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.758796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:07.780717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.782710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.788195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.790265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.790583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.791125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.791899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.799437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.799672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac4d42-89d5-4d48-a923-c868bde62bb2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.805856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.807079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.807432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.807781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] PATH_INFO: `/v3/projects/d010ada004d44207b7e168f345ecbc8a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.809062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d010ada004d44207b7e168f345ecbc8a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.811360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.833723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 941/1905] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:07 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:07.839296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.842187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.868032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f342a8-f30c-47cc-bfcf-01ca93a923f9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.880034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd010ada004d44207b7e168f345ecbc8a', 'request_id': 'req-b4ec1467-4f0a-41a6-8779-59a045f248ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.889239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd010ada004d44207b7e168f345ecbc8a', 'request_id': 'req-b4ec1467-4f0a-41a6-8779-59a045f248ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.900538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd010ada004d44207b7e168f345ecbc8a', 'request_id': 'req-b4ec1467-4f0a-41a6-8779-59a045f248ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.916970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] The token cache is being invalidate because project d010ada004d44207b7e168f345ecbc8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:07.917478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.918238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b4ec1467-4f0a-41a6-8779-59a045f248ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:07.926662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 965/1906] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:07 2022] DELETE /identity/v3/projects/d010ada004d44207b7e168f345ecbc8a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:07.928061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5066f2b-b3df-4cd9-8cf4-e51f1e0d6318 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.928368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5066f2b-b3df-4cd9-8cf4-e51f1e0d6318 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.928605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5066f2b-b3df-4cd9-8cf4-e51f1e0d6318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.951586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.952288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.952569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.952838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.953485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.954926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b5066f2b-b3df-4cd9-8cf4-e51f1e0d6318 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:07.955860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.957921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5066f2b-b3df-4cd9-8cf4-e51f1e0d6318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.975642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 942/1907] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:07.975642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:07.975950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 966/1908] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:07.976032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.978527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9c6f-52c3-44f2-b39d-16fa79dd2b49 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.978953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9c6f-52c3-44f2-b39d-16fa79dd2b49 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.979229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9c6f-52c3-44f2-b39d-16fa79dd2b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.989406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806097e-dde8-4671-bcc3-df3d3034040a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.995143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.995852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.996188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.996550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.997313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.999170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.000671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2b4f9c6f-52c3-44f2-b39d-16fa79dd2b49 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:08.001919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b4f9c6f-52c3-44f2-b39d-16fa79dd2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.012046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 967/1909] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:08.016284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f9c6f-52c3-44f2-b39d-16fa79dd2b49 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.016725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 943/1910] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:08.021839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.027144 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f9c6f-52c3-44f2-b39d-16fa79dd2b49 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.039809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.041201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85526cb-55ab-47de-9135-4741441e80c0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.042514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.042796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.043069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.044058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:08.044392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.046261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.047021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.047782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.048066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.048315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] PATH_INFO: `/v3/projects/82ec24e8ff324da49dc0893752b752bb/users/d23d14149fba41669b0fc278c47b2052/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.064048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 968/1911] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:08.065396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.067229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.070735 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ec24e8ff324da49dc0893752b752bb, user_id=d23d14149fba41669b0fc278c47b2052, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.073535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.081749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc9486-d7ad-41fc-9c42-b48f371f002d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.099611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 944/1912] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:08 2022] PUT /identity/v3/projects/82ec24e8ff324da49dc0893752b752bb/users/d23d14149fba41669b0fc278c47b2052/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:08.100608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.144188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.144836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.145278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.145551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] PATH_INFO: `/v3/projects/f4dce6088eb042cc96db922402fde842` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.146761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4dce6088eb042cc96db922402fde842)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.149023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.174555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3a345-bd5b-4334-aa12-b7e06657efda admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.179964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f0798-307f-4337-a6d1-6cf0df870941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.180534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f0798-307f-4337-a6d1-6cf0df870941 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.180822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f0798-307f-4337-a6d1-6cf0df870941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.181183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f0798-307f-4337-a6d1-6cf0df870941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.181767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f0798-307f-4337-a6d1-6cf0df870941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.183776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f0798-307f-4337-a6d1-6cf0df870941 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.206972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:08.215220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 945/1913] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:08.216355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56e24-8fbf-4ab8-b91d-74280a6855bf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.216884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56e24-8fbf-4ab8-b91d-74280a6855bf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.217293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56e24-8fbf-4ab8-b91d-74280a6855bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.224365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4dce6088eb042cc96db922402fde842', 'request_id': 'req-10513377-2c96-4aaf-bc85-f727c383a23b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.232956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4dce6088eb042cc96db922402fde842', 'request_id': 'req-10513377-2c96-4aaf-bc85-f727c383a23b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.247825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4dce6088eb042cc96db922402fde842', 'request_id': 'req-10513377-2c96-4aaf-bc85-f727c383a23b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.249021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a6b56e24-8fbf-4ab8-b91d-74280a6855bf None None] MFA Rules not processed for user `d23d14149fba41669b0fc278c47b2052`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:08.250271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a6b56e24-8fbf-4ab8-b91d-74280a6855bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.259777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:08.260305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] The token cache is being invalidate because project f4dce6088eb042cc96db922402fde842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:08.260605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.261308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.263522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 969/1914] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:08 2022] DELETE /identity/v3/projects/f4dce6088eb042cc96db922402fde842 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:08.267721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.269416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.330510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 946/1915] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:08.332868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.333390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.333698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.357343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:08.358580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.361974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10513377-2c96-4aaf-bc85-f727c383a23b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.436213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 947/1916] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:08.436408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b4a8e-4751-4979-84f8-615cea3d85c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.436985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4a8e-4751-4979-84f8-615cea3d85c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.437180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.437341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4a8e-4751-4979-84f8-615cea3d85c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.437879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4a8e-4751-4979-84f8-615cea3d85c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.438661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b4a8e-4751-4979-84f8-615cea3d85c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.438760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.440762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b4a8e-4751-4979-84f8-615cea3d85c5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.470336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 970/1917] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:08.472603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344a802-d759-4279-9818-92260af7def8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.473260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344a802-d759-4279-9818-92260af7def8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.473586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344a802-d759-4279-9818-92260af7def8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.495677 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c344a802-d759-4279-9818-92260af7def8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:08.497035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c344a802-d759-4279-9818-92260af7def8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.506578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 971/1918] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:08.510627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344a802-d759-4279-9818-92260af7def8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.515313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b64f7-3d42-404c-94fc-45d3d8e5ddbb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.520431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.521184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.521512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344a802-d759-4279-9818-92260af7def8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.521744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.521883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.526572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.527185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.527464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.527737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.529020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.530675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.532432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.534468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.542033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 972/1919] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:08.542954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.548799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 948/1920] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:08.549790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.553085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26bbfa-0f50-4782-ac84-ff5ebe88b5bf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.559244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.559865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.560149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.560422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.561077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:08.561382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.563047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.564733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ca48-e37f-40eb-af3e-8dd83f255f3b service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.572511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.573139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.573421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.573719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.575174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.576329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.582827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 973/1921] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:08.583942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.586173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.588008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 949/1922] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:08.589189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.599724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba23c1-1614-4131-b5bb-daeadd247406 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.603494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dbaf3-9915-4bbb-b6ff-0557c26eef89 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.604676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.605314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.605584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.605894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.606535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.609062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.609972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.610956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.611422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.611699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] PATH_INFO: `/v3/users/d23d14149fba41669b0fc278c47b2052` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.613016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23d14149fba41669b0fc278c47b2052)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.615045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.641243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:08.648766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 950/1923] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:08.649818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.672249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23d14149fba41669b0fc278c47b2052'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.679414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:08.686130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23d14149fba41669b0fc278c47b2052'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.694526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23d14149fba41669b0fc278c47b2052'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.695243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23d14149fba41669b0fc278c47b2052'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.702944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23d14149fba41669b0fc278c47b2052'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:08.709657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 974/1924] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:08 2022] DELETE /identity/v3/users/d23d14149fba41669b0fc278c47b2052 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:08.710500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.711856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.720154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd0f1-83fd-4997-82ce-1127abc56bf8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.780914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.781502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.781835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.782097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] PATH_INFO: `/v3/projects/be0b1a7fb7b549039e364661136b5e11/users/7203443fe2b444abbfc5cd8f71ac0da6/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.783039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff522332-a643-41b2-b53d-365ef85a6051 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.788578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4f8fb-0d00-4930-ada2-080c40cdf333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.789191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4f8fb-0d00-4930-ada2-080c40cdf333 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.789524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4f8fb-0d00-4930-ada2-080c40cdf333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.789844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4f8fb-0d00-4930-ada2-080c40cdf333 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.790472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4f8fb-0d00-4930-ada2-080c40cdf333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.792407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4f8fb-0d00-4930-ada2-080c40cdf333 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.797248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0b1a7fb7b549039e364661136b5e11, user_id=7203443fe2b444abbfc5cd8f71ac0da6, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.800615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.816886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 975/1925] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:08.817601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 951/1926] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:08 2022] PUT /identity/v3/projects/be0b1a7fb7b549039e364661136b5e11/users/7203443fe2b444abbfc5cd8f71ac0da6/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:08.821845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7da0fd-2038-46d8-85c2-14717873cd0f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.821845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7da0fd-2038-46d8-85c2-14717873cd0f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.821845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7da0fd-2038-46d8-85c2-14717873cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.827705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.838398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cf7da0fd-2038-46d8-85c2-14717873cd0f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:08.839616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf7da0fd-2038-46d8-85c2-14717873cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.894572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc026-02a3-4aee-b3fa-fe98974b5f8f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.899766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.900410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.900691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.900981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.901627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:08.903602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:08.905077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 976/1927] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:08.906383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98ba2f-05a2-474e-97c6-098ce560e845 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.906699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98ba2f-05a2-474e-97c6-098ce560e845 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.906940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98ba2f-05a2-474e-97c6-098ce560e845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:08.915878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 952/1928] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:08.916603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:08.920272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.938069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0a98ba2f-05a2-474e-97c6-098ce560e845 None None] MFA Rules not processed for user `7203443fe2b444abbfc5cd8f71ac0da6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:08.939294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0a98ba2f-05a2-474e-97c6-098ce560e845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:08.960134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:08.987004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:08.991298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e56ca-ac92-406e-a362-ce04c0b99186 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:08.996163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:08.997352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:08.997632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:08.997964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.008572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.010070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.018056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 977/1929] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:09.018809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98ba2f-05a2-474e-97c6-098ce560e845 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.024699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 953/1930] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:09.025273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.026840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:09.029702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98ba2f-05a2-474e-97c6-098ce560e845 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.034884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.035438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.035725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.036000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.036658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.038371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.042174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cdd6e-6220-4793-bff1-296a6c34d8f3 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.051440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.051440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.052214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 978/1931] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:09.052405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.052405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.052405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.054447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.055186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.065373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d11d-cfe8-4500-be7c-ed33a622a1bf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.071767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a701583-2e8d-4397-99ec-1cd96322f2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.072613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701583-2e8d-4397-99ec-1cd96322f2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.072892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701583-2e8d-4397-99ec-1cd96322f2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.073172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701583-2e8d-4397-99ec-1cd96322f2e3 admin admin] PATH_INFO: `/v3/projects/b2d8c64afff440919c4e5b6cd66494de/users/601a92ab01324b53b9952df4d225cc4b/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.090817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 954/1932] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:09 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:09.092439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a701583-2e8d-4397-99ec-1cd96322f2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d8c64afff440919c4e5b6cd66494de, user_id=601a92ab01324b53b9952df4d225cc4b, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.094441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.095253 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a701583-2e8d-4397-99ec-1cd96322f2e3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.114864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 979/1933] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:09 2022] PUT /identity/v3/projects/b2d8c64afff440919c4e5b6cd66494de/users/601a92ab01324b53b9952df4d225cc4b/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:09.127981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.128687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.129151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.165231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] MFA Rules not processed for user `601a92ab01324b53b9952df4d225cc4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:09.166499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:09.178654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16bb80-76bc-47f5-b951-6ed543489c30 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.268729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 980/1934] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:09.284209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.286331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:09.295331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.296336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.296637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.296909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.298453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.302386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.343178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 955/1935] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:09 2022] POST /identity/v3/users => generated 299 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:09.351468 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.370496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0ac10-9024-4e5a-a2b1-f406c27ccf38 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.376419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.377917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.378285 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.378581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.379248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.382161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.427664 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 956/1936] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:09.430538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.435284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d980bd-3e97-4fa4-aed2-cd73114df313 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.439569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.441124 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.441491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.442113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.442218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f7281-6010-4b0c-8e8a-7a773b898cc8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.447490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.448168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.448589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.448869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] PATH_INFO: `/v3/projects/315aab8346054060a15332916149161a/users/9a6f4bdbb9624310971d4f19dbb48541/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.454800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.456759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.468415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=315aab8346054060a15332916149161a, user_id=9a6f4bdbb9624310971d4f19dbb48541, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.469991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 981/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:07:09.472014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.490080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 957/1938] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:09 2022] PUT /identity/v3/projects/315aab8346054060a15332916149161a/users/9a6f4bdbb9624310971d4f19dbb48541/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:09.494114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.507293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e5972-6e18-4919-bac9-4bcfe8fd3117 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.523573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:09.581375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f0296-2b65-46c6-9500-1f650b547f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.582632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f0296-2b65-46c6-9500-1f650b547f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.583073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f0296-2b65-46c6-9500-1f650b547f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.583360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f0296-2b65-46c6-9500-1f650b547f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.584187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f0296-2b65-46c6-9500-1f650b547f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.586459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f0296-2b65-46c6-9500-1f650b547f76 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.600421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 982/1939] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:09.604283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.618762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871160f6-bb88-49f2-a6f1-838c540086bb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:09.623695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:09.624488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.624843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.625340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] PATH_INFO: `/v3/projects/315aab8346054060a15332916149161a/users/9a6f4bdbb9624310971d4f19dbb48541/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.627368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315aab8346054060a15332916149161a, user_id=9a6f4bdbb9624310971d4f19dbb48541, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:09.630248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:09.643108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 958/1940] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:09 2022] PUT /identity/v3/projects/315aab8346054060a15332916149161a/users/9a6f4bdbb9624310971d4f19dbb48541/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:09.655259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.655578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.655825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.700714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] MFA Rules not processed for user `9a6f4bdbb9624310971d4f19dbb48541`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:09.702413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:09.816316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 983/1941] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:09.832719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.834542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:09.897069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:09.898880 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:09.932620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:10.014179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d79c0c-dcfe-42eb-8627-fd3b0fb4a189 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.014667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceabe96-5811-404d-81a3-18de72af3ea5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.019653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.020439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.020728 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.021006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.022382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.023085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.023450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.023728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] PATH_INFO: `/v3/projects/82ec24e8ff324da49dc0893752b752bb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.025629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ec24e8ff324da49dc0893752b752bb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.027787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.033632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.043809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.056732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 959/1942] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:10.083856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:10.090422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.093560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ec24e8ff324da49dc0893752b752bb', 'request_id': 'req-e97b8125-ed16-44ae-9d48-0543b1e5960a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.100735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ec24e8ff324da49dc0893752b752bb', 'request_id': 'req-e97b8125-ed16-44ae-9d48-0543b1e5960a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.102230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef06c4-a3b4-4c27-b518-1f0eb02911ed service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.109170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ec24e8ff324da49dc0893752b752bb', 'request_id': 'req-e97b8125-ed16-44ae-9d48-0543b1e5960a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.121822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] The token cache is being invalidate because project 82ec24e8ff324da49dc0893752b752bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:10.122115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.122767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e97b8125-ed16-44ae-9d48-0543b1e5960a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.127789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 984/1943] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:09 2022] DELETE /identity/v3/projects/82ec24e8ff324da49dc0893752b752bb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:10.129094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad054354-d58d-4de7-9d12-46c47aada49f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.129379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad054354-d58d-4de7-9d12-46c47aada49f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.129654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad054354-d58d-4de7-9d12-46c47aada49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.150505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ad054354-d58d-4de7-9d12-46c47aada49f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:10.151715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ad054354-d58d-4de7-9d12-46c47aada49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.170908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.171558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.171906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.172186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] PATH_INFO: `/v3/users/7203443fe2b444abbfc5cd8f71ac0da6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.173470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7203443fe2b444abbfc5cd8f71ac0da6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.175535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.228822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 985/1944] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:10.230633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7203443fe2b444abbfc5cd8f71ac0da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.231009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718591f-bc43-41a7-ab9d-32f79eca3a00 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.231303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718591f-bc43-41a7-ab9d-32f79eca3a00 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.231585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718591f-bc43-41a7-ab9d-32f79eca3a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.240465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7203443fe2b444abbfc5cd8f71ac0da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.253437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5718591f-bc43-41a7-ab9d-32f79eca3a00 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:10.254255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7203443fe2b444abbfc5cd8f71ac0da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.254715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5718591f-bc43-41a7-ab9d-32f79eca3a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.255272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7203443fe2b444abbfc5cd8f71ac0da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.266912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 986/1945] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:10.268105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718591f-bc43-41a7-ab9d-32f79eca3a00 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.269503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15abb205-0cc5-4a48-a8c4-a598c8714c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7203443fe2b444abbfc5cd8f71ac0da6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:10.277570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 960/1946] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:10 2022] DELETE /identity/v3/users/7203443fe2b444abbfc5cd8f71ac0da6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:10.281116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718591f-bc43-41a7-ab9d-32f79eca3a00 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.282590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.282924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.283151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.304800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:10.305637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.367727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:10.379137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.379826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.380117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.380551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.381349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:10.381711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.383415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.402325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 961/1947] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:10.409656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.409866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 987/1948] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:10.414099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.414318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.424902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3fde5-a78d-4d48-bbb8-0b5a007d16f1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.432780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.433542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.433908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.434390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.435069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.436995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.483763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 988/1949] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:10 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:10.485057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.501295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ca26b-2596-4825-be9e-2e24de833b2e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.535933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:10.567680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.568420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.568703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.568973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.570373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ab762-c9b0-492d-b576-0f8a9dd27615 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.578587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.579173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.579740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.580198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.580640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.580921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.581898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.584040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.592127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 962/1950] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:07:10.613780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 989/1951] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:10 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:10.615828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.629504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f4e0d-8cfe-4067-bd18-88b19afb6f2c service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.635177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.635867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.636290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.636564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.637199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.638848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.653847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 963/1952] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:10.656481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.670649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb67d-b193-4b9b-aa4e-67aec6cd6371 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.677702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde826c-2ddc-4050-a35f-d85caa8861b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.678379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde826c-2ddc-4050-a35f-d85caa8861b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.678618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde826c-2ddc-4050-a35f-d85caa8861b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.678926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde826c-2ddc-4050-a35f-d85caa8861b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.679526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde826c-2ddc-4050-a35f-d85caa8861b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.681391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde826c-2ddc-4050-a35f-d85caa8861b0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.702087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 990/1953] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:10.702861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.715286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f9bbf-08b8-47f6-ab52-06b12c4e1af9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.720135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.720868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.721128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.721416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] PATH_INFO: `/v3/projects/79a70ef4410942feb4493a0503d4cc6f/users/0aca9b490c57472289a0e921d83997be/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.739945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a70ef4410942feb4493a0503d4cc6f, user_id=0aca9b490c57472289a0e921d83997be, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.742937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:10.760747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 964/1954] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:10 2022] PUT /identity/v3/projects/79a70ef4410942feb4493a0503d4cc6f/users/0aca9b490c57472289a0e921d83997be/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:10.773944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.779332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.779332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.810202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] MFA Rules not processed for user `0aca9b490c57472289a0e921d83997be`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:10.811375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.880438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 991/1955] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:10.892455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.894602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.913549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:10.915790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.973012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c7fd-10c4-48ed-82e2-103f7862611b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:10.977165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:10.978095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:10.978313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:10.978657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.989472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:10.990963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:11.008364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 965/1956] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:11.022805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7bea5-af38-412a-9fdd-6c6ddfacf9b1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:11.028108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:11.029844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:11.030324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:11.030742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] PATH_INFO: `/v3/projects/be0b1a7fb7b549039e364661136b5e11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:11.042723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0b1a7fb7b549039e364661136b5e11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:11.044875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:11.116072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:11.121730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0b1a7fb7b549039e364661136b5e11', 'request_id': 'req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.130224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0b1a7fb7b549039e364661136b5e11', 'request_id': 'req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.140063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0b1a7fb7b549039e364661136b5e11', 'request_id': 'req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.155955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] The token cache is being invalidate because project be0b1a7fb7b549039e364661136b5e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:11.156640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.157329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.159033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 992/1957] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:10 2022] DELETE /identity/v3/projects/be0b1a7fb7b549039e364661136b5e11 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:11.163242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:11.174170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553abef1-bc2b-4e02-958b-7005dff8e99e service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:11.177750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:11.239295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:11.239492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:11.240037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:11.240382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] PATH_INFO: `/v3/users/9a6f4bdbb9624310971d4f19dbb48541` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:11.241759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6f4bdbb9624310971d4f19dbb48541)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:11.244120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:11.326832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6f4bdbb9624310971d4f19dbb48541'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.338447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6f4bdbb9624310971d4f19dbb48541'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.356752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6f4bdbb9624310971d4f19dbb48541'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.363241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6f4bdbb9624310971d4f19dbb48541'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.374654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec46a11c-6a3f-4d6e-a612-0c30589176de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6f4bdbb9624310971d4f19dbb48541'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.386856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 966/1958] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:11 2022] DELETE /identity/v3/users/9a6f4bdbb9624310971d4f19dbb48541 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:11.691828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:11.693384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:11.708236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979bef7-d8f4-4efc-8ec3-6ec407bdd937 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:11.773301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:11.774208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:11.774489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:11.774761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] PATH_INFO: `/v3/projects/315aab8346054060a15332916149161a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:11.775916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=315aab8346054060a15332916149161a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:11.777935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:11.837838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315aab8346054060a15332916149161a', 'request_id': 'req-b0f8388a-03e7-4170-b264-ea8692cc0e83'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.844796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315aab8346054060a15332916149161a', 'request_id': 'req-b0f8388a-03e7-4170-b264-ea8692cc0e83'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.853975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315aab8346054060a15332916149161a', 'request_id': 'req-b0f8388a-03e7-4170-b264-ea8692cc0e83'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.865826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] The token cache is being invalidate because project 315aab8346054060a15332916149161a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:11.866117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.866759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b0f8388a-03e7-4170-b264-ea8692cc0e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:11.868303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 993/1959] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:11 2022] DELETE /identity/v3/projects/315aab8346054060a15332916149161a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:11.873386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16029136-6479-4b1d-b3f2-896f7b86dc55 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:11.873743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16029136-6479-4b1d-b3f2-896f7b86dc55 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:11.874021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16029136-6479-4b1d-b3f2-896f7b86dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:11.894884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-16029136-6479-4b1d-b3f2-896f7b86dc55 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:11.896104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-16029136-6479-4b1d-b3f2-896f7b86dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:11.957411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 967/1960] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:11.965524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741adcc9-b144-403e-b169-671e4a6a4590 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:11.965882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741adcc9-b144-403e-b169-671e4a6a4590 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:11.966194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741adcc9-b144-403e-b169-671e4a6a4590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:11.987194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-741adcc9-b144-403e-b169-671e4a6a4590 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:11.988425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-741adcc9-b144-403e-b169-671e4a6a4590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:11.997550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 994/1961] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:12.002147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16029136-6479-4b1d-b3f2-896f7b86dc55 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.012711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16029136-6479-4b1d-b3f2-896f7b86dc55 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.018958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.019585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.019858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.020128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.020769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:12.021069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.022704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.036283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 968/1962] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:12.040889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741adcc9-b144-403e-b169-671e4a6a4590 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.050866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741adcc9-b144-403e-b169-671e4a6a4590 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.055996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.056605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.056866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.057293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.057942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.059845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.089706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 995/1963] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:12 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:12.095896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.106774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f01b16-a060-452f-a7d4-e90ebc74dd94 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.117102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:12.168845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.169465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.169779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.170059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.170697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.172629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.195955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 969/1964] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:12 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:12.200391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.212052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50790c09-64db-4943-8999-7cb321b9d48e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.217592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.218460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.218900 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.219190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.220023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.221792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.235844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 996/1965] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:12.237940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.251778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90ce2-ff9c-4b00-a3a5-0e7cb2ed71d8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.257141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa466ca-522f-4645-9549-9c3fcfdb2f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.257851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa466ca-522f-4645-9549-9c3fcfdb2f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.258133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa466ca-522f-4645-9549-9c3fcfdb2f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.258407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa466ca-522f-4645-9549-9c3fcfdb2f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.259087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa466ca-522f-4645-9549-9c3fcfdb2f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.260759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa466ca-522f-4645-9549-9c3fcfdb2f89 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.272732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 970/1966] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:07:12.277340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.287524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808461c-5c09-4009-867c-8383163940ba admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.296394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.297077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.297324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.297590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] PATH_INFO: `/v3/projects/70dfe075be9c42eda96408403a4e01fd/users/40d5782b8662407d8d6da22b1c00fd2c/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.313647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70dfe075be9c42eda96408403a4e01fd, user_id=40d5782b8662407d8d6da22b1c00fd2c, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.316643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.328061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:12.333279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 997/1967] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:12 2022] PUT /identity/v3/projects/70dfe075be9c42eda96408403a4e01fd/users/40d5782b8662407d8d6da22b1c00fd2c/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:12.347110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d74f62-4efc-492d-945a-350c216bb1a3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.347534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d74f62-4efc-492d-945a-350c216bb1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.348039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d74f62-4efc-492d-945a-350c216bb1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.391303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e7d74f62-4efc-492d-945a-350c216bb1a3 None None] MFA Rules not processed for user `40d5782b8662407d8d6da22b1c00fd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:12.393433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e7d74f62-4efc-492d-945a-350c216bb1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:12.479877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 971/1968] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:12.484177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.551331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bce3a-95d9-4dc2-bdb0-8d59f2a613c3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.556955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.557589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.557906 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.558184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.558824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.560770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.591294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 998/1969] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:12 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:12.595650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d74f62-4efc-492d-945a-350c216bb1a3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.607088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d74f62-4efc-492d-945a-350c216bb1a3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.667561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.668173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.668451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.668726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.669370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.671364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.694508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 972/1970] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:12 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:12.699128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.710485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0260d-310a-4198-945a-eabebc9ffb85 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.715286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.715896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.716178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.716450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.717089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.718743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.730026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 999/1971] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:12.734684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.746364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace3ed-46f4-45d5-becd-eb7815bc0bc6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.751969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.752642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.753118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.753514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] PATH_INFO: `/v3/projects/6dd92a4fd7cf48f8b2e4651cbb2f7ade/users/75578d4e567f47c7b18ae5ce9247be21/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.762823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:12.771626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd92a4fd7cf48f8b2e4651cbb2f7ade, user_id=75578d4e567f47c7b18ae5ce9247be21, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.774528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.789763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 973/1972] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:12 2022] PUT /identity/v3/projects/6dd92a4fd7cf48f8b2e4651cbb2f7ade/users/75578d4e567f47c7b18ae5ce9247be21/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:12.794005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.806653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e07ab-dc40-4b72-96af-048f168e288e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.874664 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367ba0b-3a43-43ac-8e38-cd3244462bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.875251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367ba0b-3a43-43ac-8e38-cd3244462bca admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.875533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367ba0b-3a43-43ac-8e38-cd3244462bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.875800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367ba0b-3a43-43ac-8e38-cd3244462bca admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.876487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367ba0b-3a43-43ac-8e38-cd3244462bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.878965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367ba0b-3a43-43ac-8e38-cd3244462bca admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.893795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1000/1973] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:12.896312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.910518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597394a-52e5-46b3-9923-c897b93ad9db admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:12.917558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:12.918290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.918655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.918823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] PATH_INFO: `/v3/projects/6dd92a4fd7cf48f8b2e4651cbb2f7ade/users/75578d4e567f47c7b18ae5ce9247be21/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.921049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd92a4fd7cf48f8b2e4651cbb2f7ade, user_id=75578d4e567f47c7b18ae5ce9247be21, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:12.926737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:12.941649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 974/1974] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:12 2022] PUT /identity/v3/projects/6dd92a4fd7cf48f8b2e4651cbb2f7ade/users/75578d4e567f47c7b18ae5ce9247be21/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:12.953051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6f8a8-cba0-449a-a791-b6b3179227eb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.953317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6f8a8-cba0-449a-a791-b6b3179227eb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.953593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6f8a8-cba0-449a-a791-b6b3179227eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.989715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-09f6f8a8-cba0-449a-a791-b6b3179227eb None None] MFA Rules not processed for user `75578d4e567f47c7b18ae5ce9247be21`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:12.991109 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-09f6f8a8-cba0-449a-a791-b6b3179227eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:13.064046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:13.074353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1001/1975] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:13.086394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:13.088778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:13.184203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e5539-4852-402d-93c8-88a3d8f9b4c9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:13.188845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c753445-c854-49ae-94ae-c680ec1f4564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:13.189565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c753445-c854-49ae-94ae-c680ec1f4564 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:13.190062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c753445-c854-49ae-94ae-c680ec1f4564 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:13.190519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c753445-c854-49ae-94ae-c680ec1f4564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:13.203548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c753445-c854-49ae-94ae-c680ec1f4564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:13.205010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c753445-c854-49ae-94ae-c680ec1f4564 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:13.216083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 975/1976] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:13.779710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6f8a8-cba0-449a-a791-b6b3179227eb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:13.900727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6f8a8-cba0-449a-a791-b6b3179227eb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:13.907140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:13.907879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:13.908245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:13.908528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] PATH_INFO: `/v3/users/40d5782b8662407d8d6da22b1c00fd2c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:13.909919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d5782b8662407d8d6da22b1c00fd2c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:13.912189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:14.009534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d5782b8662407d8d6da22b1c00fd2c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:14.038378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d5782b8662407d8d6da22b1c00fd2c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:14.053101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d5782b8662407d8d6da22b1c00fd2c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:14.055513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d5782b8662407d8d6da22b1c00fd2c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:14.067203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d5782b8662407d8d6da22b1c00fd2c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:14.079670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1002/1977] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:13 2022] DELETE /identity/v3/users/40d5782b8662407d8d6da22b1c00fd2c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:14.083341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:14.083878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:14.084821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:14.120042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:14.121852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:14.214690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 976/1978] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:14.228345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:14.232273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:14.392074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac674f-db14-4f8c-8155-dc21b997f35e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:14.402035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:14.402685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:14.402959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:14.403222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:14.413570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:14.414891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:14.429776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1003/1979] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:15.149547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:15.151299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:15.164763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037fd1b-0bc9-49bd-9c34-05b31c843d31 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:15.170184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:15.171429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:15.171429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:15.172463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] PATH_INFO: `/v3/projects/70dfe075be9c42eda96408403a4e01fd` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:15.175544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70dfe075be9c42eda96408403a4e01fd)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:15.179510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:15.254831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70dfe075be9c42eda96408403a4e01fd', 'request_id': 'req-8884132c-ce95-47bb-8ec4-f8c142679e63'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.260842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70dfe075be9c42eda96408403a4e01fd', 'request_id': 'req-8884132c-ce95-47bb-8ec4-f8c142679e63'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.267550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70dfe075be9c42eda96408403a4e01fd', 'request_id': 'req-8884132c-ce95-47bb-8ec4-f8c142679e63'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.279878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:15.284297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] The token cache is being invalidate because project 70dfe075be9c42eda96408403a4e01fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:15.285103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.286054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.287848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 977/1980] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:15 2022] DELETE /identity/v3/projects/70dfe075be9c42eda96408403a4e01fd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:15.292060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:15.310949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc391959-4022-4f2c-b91c-77e433ca8207 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:15.341802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:15.437333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:15.438030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:15.438316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:15.438591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] PATH_INFO: `/v3/users/75578d4e567f47c7b18ae5ce9247be21` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:15.440031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75578d4e567f47c7b18ae5ce9247be21)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:15.442285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:15.505356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75578d4e567f47c7b18ae5ce9247be21'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.521559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75578d4e567f47c7b18ae5ce9247be21'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.534730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75578d4e567f47c7b18ae5ce9247be21'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.536432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75578d4e567f47c7b18ae5ce9247be21'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.550174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b7544c68-bc41-49df-ab6c-a8f6f8454fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75578d4e567f47c7b18ae5ce9247be21'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:15.562849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1004/1981] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:15 2022] DELETE /identity/v3/users/75578d4e567f47c7b18ae5ce9247be21 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:16.312741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:16.314360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:16.330408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884132c-ce95-47bb-8ec4-f8c142679e63 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:16.455103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:16.455670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.456833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.457515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] PATH_INFO: `/v3/projects/6dd92a4fd7cf48f8b2e4651cbb2f7ade` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.459746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd92a4fd7cf48f8b2e4651cbb2f7ade)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:16.462132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:16.554748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd92a4fd7cf48f8b2e4651cbb2f7ade', 'request_id': 'req-d85df46e-ab51-4049-979f-5ddeab93a013'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:16.563427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd92a4fd7cf48f8b2e4651cbb2f7ade', 'request_id': 'req-d85df46e-ab51-4049-979f-5ddeab93a013'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:16.578775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd92a4fd7cf48f8b2e4651cbb2f7ade', 'request_id': 'req-d85df46e-ab51-4049-979f-5ddeab93a013'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:16.601096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] The token cache is being invalidate because project 6dd92a4fd7cf48f8b2e4651cbb2f7ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:16.601351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:16.602715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d85df46e-ab51-4049-979f-5ddeab93a013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:16.606850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 978/1982] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:16 2022] DELETE /identity/v3/projects/6dd92a4fd7cf48f8b2e4651cbb2f7ade => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:16.612497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc24b7f-9518-4dbd-8b52-84c66bad6ded None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.613083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc24b7f-9518-4dbd-8b52-84c66bad6ded None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.613083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc24b7f-9518-4dbd-8b52-84c66bad6ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.645933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1dc24b7f-9518-4dbd-8b52-84c66bad6ded None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:16.647739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1dc24b7f-9518-4dbd-8b52-84c66bad6ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:16.740123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1005/1983] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:16.759599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe6462-a2d6-4501-be27-0d249e6c6c27 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.759909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe6462-a2d6-4501-be27-0d249e6c6c27 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.760161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe6462-a2d6-4501-be27-0d249e6c6c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.785034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fdfe6462-a2d6-4501-be27-0d249e6c6c27 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:16.786481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fdfe6462-a2d6-4501-be27-0d249e6c6c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:16.796954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 979/1984] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:16.801211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc24b7f-9518-4dbd-8b52-84c66bad6ded None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:16.819676 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc24b7f-9518-4dbd-8b52-84c66bad6ded None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:16.839378 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:16.840011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.840472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.840743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.841548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:16.842060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:16.843909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:16.861826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1006/1985] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:16.871212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe6462-a2d6-4501-be27-0d249e6c6c27 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:16.883872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe6462-a2d6-4501-be27-0d249e6c6c27 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:16.888731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:16.889360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.890111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.890111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.890616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:16.894177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:16.930577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 980/1986] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:16 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:16.934820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:16.945395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae78d65-bebe-4088-b3b3-1a5bfb7819bc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:16.969016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.014912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.015575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.015859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.016137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.017226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.019384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.047427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1007/1987] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:16 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:17.049127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.060576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a832c0-1502-4bea-a31b-c0d232a0126e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.066306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.066926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.067207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.067477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.068185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.069903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.080617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 981/1988] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:17.084791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.094860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b81271-0138-4bda-9d22-8d78fa8dd22b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.100373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0c3b6-4d43-4b5c-8c6b-04c025b54991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.100991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c3b6-4d43-4b5c-8c6b-04c025b54991 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.101269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c3b6-4d43-4b5c-8c6b-04c025b54991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.101539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c3b6-4d43-4b5c-8c6b-04c025b54991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.102213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0c3b6-4d43-4b5c-8c6b-04c025b54991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.103815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0c3b6-4d43-4b5c-8c6b-04c025b54991 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.117033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1008/1989] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:17.120721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.135808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b277e-f077-45f5-8127-7eb3f28c2aef admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.140655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.141334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.141994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.142212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] PATH_INFO: `/v3/projects/44a68b82c03649a58b251f892243c270/users/a4cc5f69ff7445a69a9cf184ce3ba79f/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.158259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a68b82c03649a58b251f892243c270, user_id=a4cc5f69ff7445a69a9cf184ce3ba79f, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.161183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.176404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 982/1990] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:17 2022] PUT /identity/v3/projects/44a68b82c03649a58b251f892243c270/users/a4cc5f69ff7445a69a9cf184ce3ba79f/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:17.186918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d070-a5b3-4e23-aeb5-b3bc485cd1e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.187348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d070-a5b3-4e23-aeb5-b3bc485cd1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.187481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d070-a5b3-4e23-aeb5-b3bc485cd1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.219487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e144d070-a5b3-4e23-aeb5-b3bc485cd1e9 None None] MFA Rules not processed for user `a4cc5f69ff7445a69a9cf184ce3ba79f`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:17.220611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e144d070-a5b3-4e23-aeb5-b3bc485cd1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:17.291753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1009/1991] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:17.296295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.331512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.370296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bee207-22e7-4ffa-8856-d2379346a637 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.375063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.375687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.376016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.376316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.376903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.381143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.420992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 983/1992] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:17 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:17.427133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144d070-a5b3-4e23-aeb5-b3bc485cd1e9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.441625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144d070-a5b3-4e23-aeb5-b3bc485cd1e9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.514668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.515320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.515599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.515888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.516509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.518848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.542360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1010/1993] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:17 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:17.546783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.557534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1d94-4912-48d9-841c-4fa3631f93c1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.566659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.566659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.566659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.566659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.566989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.566989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.577242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 984/1994] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:17.581927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.592504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc2d1f-321e-4765-83cb-28685b69e4ec admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.597452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.598059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.598334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.598606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] PATH_INFO: `/v3/projects/b8e6562d60fd4eab875a848cb89fb941/users/4291202c7c8a4c61b31db9c4e4ed0bfe/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.615631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e6562d60fd4eab875a848cb89fb941, user_id=4291202c7c8a4c61b31db9c4e4ed0bfe, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.618362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.627375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.634189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1011/1995] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:17 2022] PUT /identity/v3/projects/b8e6562d60fd4eab875a848cb89fb941/users/4291202c7c8a4c61b31db9c4e4ed0bfe/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:17.638486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.651179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5b6fb-206a-4f62-adc4-0d11cb3ea517 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.715536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa40cb3-942b-41dd-9321-c6373fdd719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.716192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa40cb3-942b-41dd-9321-c6373fdd719f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.716470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa40cb3-942b-41dd-9321-c6373fdd719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.716744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa40cb3-942b-41dd-9321-c6373fdd719f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.717370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa40cb3-942b-41dd-9321-c6373fdd719f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.719024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa40cb3-942b-41dd-9321-c6373fdd719f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.731140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 985/1996] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:17.736710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.748055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec120385-6743-4002-b95b-f4ff859cc049 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.753520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.754238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.754546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.754813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] PATH_INFO: `/v3/projects/b8e6562d60fd4eab875a848cb89fb941/users/4291202c7c8a4c61b31db9c4e4ed0bfe/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.756964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e6562d60fd4eab875a848cb89fb941, user_id=4291202c7c8a4c61b31db9c4e4ed0bfe, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.759660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.770839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1012/1997] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:17 2022] PUT /identity/v3/projects/b8e6562d60fd4eab875a848cb89fb941/users/4291202c7c8a4c61b31db9c4e4ed0bfe/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:17.782743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.783144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.783562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.816466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] MFA Rules not processed for user `4291202c7c8a4c61b31db9c4e4ed0bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:17.817718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:17.921823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 986/1998] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:17.940705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.942676 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:18.010016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6211a6-7c00-488e-8a7b-24ef7bf88dc2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:18.014537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2b89d-ce67-4b35-8d2a-0fcc8a149052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:18.015157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2b89d-ce67-4b35-8d2a-0fcc8a149052 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:18.015443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2b89d-ce67-4b35-8d2a-0fcc8a149052 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:18.015717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2b89d-ce67-4b35-8d2a-0fcc8a149052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:18.089297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2b89d-ce67-4b35-8d2a-0fcc8a149052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:18.090846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2b89d-ce67-4b35-8d2a-0fcc8a149052 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:18.101780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1013/1999] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:22.121929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:22.123552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:22.150969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:22.195484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafddde7-f7ff-448d-abcc-25d07d2dc7ef None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:22.201655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:22.202379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.202632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.202933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] PATH_INFO: `/v3/users/0aca9b490c57472289a0e921d83997be` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.204282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aca9b490c57472289a0e921d83997be)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:22.206336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:22.259981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aca9b490c57472289a0e921d83997be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:22.268802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aca9b490c57472289a0e921d83997be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:22.282057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aca9b490c57472289a0e921d83997be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:22.282764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aca9b490c57472289a0e921d83997be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:22.290197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aca9b490c57472289a0e921d83997be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:22.296896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 987/2000] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:22 2022] DELETE /identity/v3/users/0aca9b490c57472289a0e921d83997be => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:22.301786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.302294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.302396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.322372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:22.323591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:22.387393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1014/2001] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:22.398474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:22.400132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:22.462398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fe0c1-3c1b-4a2a-a833-ea24027c1fbd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:22.466300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269923e2-7ddc-4267-9034-5970fb12410b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:22.466974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269923e2-7ddc-4267-9034-5970fb12410b service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.467320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269923e2-7ddc-4267-9034-5970fb12410b service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.467589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269923e2-7ddc-4267-9034-5970fb12410b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.477552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269923e2-7ddc-4267-9034-5970fb12410b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:22.479028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269923e2-7ddc-4267-9034-5970fb12410b service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:22.489295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 988/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:22.924763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:22.928983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:22.954406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd9a3b-8058-472c-8697-4e1f029aa129 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:22.961111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:22.961987 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.962471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.962471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] PATH_INFO: `/v3/projects/79a70ef4410942feb4493a0503d4cc6f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.963641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a70ef4410942feb4493a0503d4cc6f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:22.965919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.019225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:23.030087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a70ef4410942feb4493a0503d4cc6f', 'request_id': 'req-96fa8728-d6c7-480e-9c63-f05572bd7f5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:23.037788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a70ef4410942feb4493a0503d4cc6f', 'request_id': 'req-96fa8728-d6c7-480e-9c63-f05572bd7f5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:23.046349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a70ef4410942feb4493a0503d4cc6f', 'request_id': 'req-96fa8728-d6c7-480e-9c63-f05572bd7f5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:23.057851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] The token cache is being invalidate because project 79a70ef4410942feb4493a0503d4cc6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:23.058118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:23.058861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-96fa8728-d6c7-480e-9c63-f05572bd7f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:23.060752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1015/2003] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:22 2022] DELETE /identity/v3/projects/79a70ef4410942feb4493a0503d4cc6f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:23.066767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8cbe-4d64-4686-97f2-45c4330fc047 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.067040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8cbe-4d64-4686-97f2-45c4330fc047 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.067309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8cbe-4d64-4686-97f2-45c4330fc047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.091810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8a7a8cbe-4d64-4686-97f2-45c4330fc047 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:23.093155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8a7a8cbe-4d64-4686-97f2-45c4330fc047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:23.139257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:23.163550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 989/2004] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:23.172699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d1f9-0645-48af-95ca-87e87d46c8e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.172986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d1f9-0645-48af-95ca-87e87d46c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.173259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d1f9-0645-48af-95ca-87e87d46c8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.193932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-97c3d1f9-0645-48af-95ca-87e87d46c8e9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:23.195183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-97c3d1f9-0645-48af-95ca-87e87d46c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:23.203657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1016/2005] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:23.208330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8cbe-4d64-4686-97f2-45c4330fc047 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.218581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8cbe-4d64-4686-97f2-45c4330fc047 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.223512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.224131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.224411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.224685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.225330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:23.225633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.227269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.241772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 990/2006] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:23.244542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3d1f9-0645-48af-95ca-87e87d46c8e9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.256267 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3d1f9-0645-48af-95ca-87e87d46c8e9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.261921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.262531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.262804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.263085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.263750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.265755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.297218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1017/2007] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:23 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:23.302008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.312397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237edea4-db33-4857-8e28-0f2aa82c09c1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.376051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.376726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.377008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.377300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.378007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.379938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.404708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 991/2008] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:23 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:23.409312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.420736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f04658-fdc5-4342-b721-010ca54b736f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.426061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.426695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.426979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.427254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.427868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.429511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.441170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1018/2009] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:23.445494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.455733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ae0ce-041b-4540-8bca-b712879b2fa1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.460737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34396271-0965-4e1c-9111-ba5fe1fd97fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.461355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396271-0965-4e1c-9111-ba5fe1fd97fb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.461639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396271-0965-4e1c-9111-ba5fe1fd97fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.461951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396271-0965-4e1c-9111-ba5fe1fd97fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.462596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34396271-0965-4e1c-9111-ba5fe1fd97fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.464220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34396271-0965-4e1c-9111-ba5fe1fd97fb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.475920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 992/2010] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:23.480423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.491330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd845b8-280e-48e2-b237-129bc3369e10 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.496790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.497487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.497780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.498084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] PATH_INFO: `/v3/projects/1f2e1474e0024ac3918f295c3e228869/users/c589c9ed14594c498eb5a2af90030231/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.515996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2e1474e0024ac3918f295c3e228869, user_id=c589c9ed14594c498eb5a2af90030231, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.519233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.533096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1019/2011] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:23 2022] PUT /identity/v3/projects/1f2e1474e0024ac3918f295c3e228869/users/c589c9ed14594c498eb5a2af90030231/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:23.544106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80dbec-6c1c-418b-ab7a-a298638c0091 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.544397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80dbec-6c1c-418b-ab7a-a298638c0091 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.544677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80dbec-6c1c-418b-ab7a-a298638c0091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.574562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ad80dbec-6c1c-418b-ab7a-a298638c0091 None None] MFA Rules not processed for user `c589c9ed14594c498eb5a2af90030231`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:23.575800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ad80dbec-6c1c-418b-ab7a-a298638c0091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:23.658624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 993/2012] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:23.676205 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.678076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:23.744865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af10d5-57b5-4f3f-9e41-aedee0760adb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.750880 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.751845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.752298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.752743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.768142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.770551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.786221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1020/2013] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:28.665557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d2126-8da1-43d5-9571-329163123275 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:28.666160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d2126-8da1-43d5-9571-329163123275 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:28.666160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d2126-8da1-43d5-9571-329163123275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:28.687197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-966d2126-8da1-43d5-9571-329163123275 None None] MFA Rules not processed for user `c589c9ed14594c498eb5a2af90030231`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:28.688436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-966d2126-8da1-43d5-9571-329163123275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:28.698352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 994/2014] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:31.172894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:31.183778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee9643-748b-44f3-8efa-09556258c5ac service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:31.188690 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19133442-dd2a-408d-9d71-5e240c5eb974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:31.189710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133442-dd2a-408d-9d71-5e240c5eb974 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:31.190173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133442-dd2a-408d-9d71-5e240c5eb974 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:31.190628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133442-dd2a-408d-9d71-5e240c5eb974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:31.192943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-19133442-dd2a-408d-9d71-5e240c5eb974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:31.313614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:31.316424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19133442-dd2a-408d-9d71-5e240c5eb974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:31.319123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19133442-dd2a-408d-9d71-5e240c5eb974 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:31.332341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1021/2015] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:38.681281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:41.339074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:45.608950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d2126-8da1-43d5-9571-329163123275 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.611089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-966d2126-8da1-43d5-9571-329163123275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:45.681663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d2126-8da1-43d5-9571-329163123275 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:45.686219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:45.686866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.687145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.687420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] PATH_INFO: `/v3/users/5a70fbdf54c24e46b7c81008808808f1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.688545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a70fbdf54c24e46b7c81008808808f1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:45.690546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:45.767617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a70fbdf54c24e46b7c81008808808f1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:45.778111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a70fbdf54c24e46b7c81008808808f1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:45.794994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a70fbdf54c24e46b7c81008808808f1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:45.795736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a70fbdf54c24e46b7c81008808808f1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:45.804051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a70fbdf54c24e46b7c81008808808f1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:45.810205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 995/2016] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:45 2022] DELETE /identity/v3/users/5a70fbdf54c24e46b7c81008808808f1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:45.814734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.815247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.815337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.838201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:45.839515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:45.919199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1022/2017] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:45.929149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.930644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:46.001955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f813b94-3734-4905-9209-472e406efe8e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:46.006628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:46.007267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.007608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.007895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.019499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:46.020744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:46.031349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 996/2018] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:46.323104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.324596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:46.338450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7505e2-65db-41d5-842d-4116ad0b857b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:46.343998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:46.345239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.345545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.346090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] PATH_INFO: `/v3/projects/329819bc48c242df813cc4e36faa88e0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.347832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] RBAC: Authorizing `identity:delete_project(project_id=329819bc48c242df813cc4e36faa88e0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:46.349522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:46.405758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329819bc48c242df813cc4e36faa88e0', 'request_id': 'req-9732697e-deda-4d3a-adb1-c047b980293d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.413327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329819bc48c242df813cc4e36faa88e0', 'request_id': 'req-9732697e-deda-4d3a-adb1-c047b980293d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.421959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329819bc48c242df813cc4e36faa88e0', 'request_id': 'req-9732697e-deda-4d3a-adb1-c047b980293d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.433132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] The token cache is being invalidate because project 329819bc48c242df813cc4e36faa88e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:46.433424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.434162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.435839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1023/2019] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:46 2022] DELETE /identity/v3/projects/329819bc48c242df813cc4e36faa88e0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:46.440481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.452048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8daf6-645f-4d87-ac15-5e77b6ce08a8 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:46.470801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:46.517421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:46.518415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.518690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.518950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] PATH_INFO: `/v3/users/b0f37bbc0f3345d1994074d9b938aacc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.520804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f37bbc0f3345d1994074d9b938aacc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:46.522826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:46.577384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f37bbc0f3345d1994074d9b938aacc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.589121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f37bbc0f3345d1994074d9b938aacc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.620651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f37bbc0f3345d1994074d9b938aacc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.621888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f37bbc0f3345d1994074d9b938aacc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.634642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f37bbc0f3345d1994074d9b938aacc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:46.647923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 997/2020] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:46 2022] DELETE /identity/v3/users/b0f37bbc0f3345d1994074d9b938aacc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:47.094446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.096876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:47.126833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732697e-deda-4d3a-adb1-c047b980293d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.257850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.258527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.258804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.259184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] PATH_INFO: `/v3/users/a4cc5f69ff7445a69a9cf184ce3ba79f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.260992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4cc5f69ff7445a69a9cf184ce3ba79f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.264610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.322023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.354660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4cc5f69ff7445a69a9cf184ce3ba79f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.391710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4cc5f69ff7445a69a9cf184ce3ba79f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.410898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4cc5f69ff7445a69a9cf184ce3ba79f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.411960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4cc5f69ff7445a69a9cf184ce3ba79f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.422951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81e53bc6-82d6-4403-a522-15fc2791f90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4cc5f69ff7445a69a9cf184ce3ba79f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.431400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.433582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:47.439426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1024/2021] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:47 2022] DELETE /identity/v3/users/a4cc5f69ff7445a69a9cf184ce3ba79f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:47.446720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.446720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.446720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.462486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26c8b6-1f32-4c0c-8632-00446eeb4d6d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.479373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:47.481251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:47.582836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.584221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.584713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.585269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] PATH_INFO: `/v3/projects/cbecfc3e7497400eac1c0e9066ed5c5f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.587684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbecfc3e7497400eac1c0e9066ed5c5f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.589850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1025/2022] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:47.591641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.604328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.606472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:47.690599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.696599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbecfc3e7497400eac1c0e9066ed5c5f', 'request_id': 'req-7d214b6e-624c-4767-90a6-04baeb12a40b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.711346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbecfc3e7497400eac1c0e9066ed5c5f', 'request_id': 'req-7d214b6e-624c-4767-90a6-04baeb12a40b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.721248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6accb-4361-4b1c-b03c-432f82b69da9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.725485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbecfc3e7497400eac1c0e9066ed5c5f', 'request_id': 'req-7d214b6e-624c-4767-90a6-04baeb12a40b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.757915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] The token cache is being invalidate because project cbecfc3e7497400eac1c0e9066ed5c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:47.758437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.759682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7d214b6e-624c-4767-90a6-04baeb12a40b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:47.761968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 998/2023] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:47 2022] DELETE /identity/v3/projects/cbecfc3e7497400eac1c0e9066ed5c5f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:47.768971 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cfca6-e20c-439f-8d45-78d053b78a1a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.769765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cfca6-e20c-439f-8d45-78d053b78a1a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.770538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cfca6-e20c-439f-8d45-78d053b78a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.831055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.832787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.832787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.833082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.834296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cc2cfca6-e20c-439f-8d45-78d053b78a1a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:47.835881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cc2cfca6-e20c-439f-8d45-78d053b78a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:47.836128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:47.861001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.957090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 999/2024] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:47.973178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc8d60-fb99-4186-b8a9-bef159f917f3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.973688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc8d60-fb99-4186-b8a9-bef159f917f3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.975825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc8d60-fb99-4186-b8a9-bef159f917f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.988545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.992777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.007393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-63fc8d60-fb99-4186-b8a9-bef159f917f3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:48.008667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-63fc8d60-fb99-4186-b8a9-bef159f917f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:48.022885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1026/2026] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:48.025013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.031515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1000/2026] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:48.047111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8440f-75c9-4c5b-bf29-410259556ded service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.074281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.075648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.076200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.076726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.078045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:48.078710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.081877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.109240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1027/2027] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:48.111276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc8d60-fb99-4186-b8a9-bef159f917f3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.130538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc8d60-fb99-4186-b8a9-bef159f917f3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.138669 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.139370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.139934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.140485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.141909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.146153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.198542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1001/2028] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:48 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:48.203064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.213738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77078c87-b613-408b-8c09-828c46b75903 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.339348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.340228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.340539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.340789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.342839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.345387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.389858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1028/2029] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:48 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:48.393268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.412611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81023c0e-33ab-461c-89d3-cf7149146c23 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.420614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.421428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.421782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.422109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.422831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.425040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.446185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1002/2030] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:07:48.451217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.495404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cee70-67a6-4e6b-a618-5abe5e14ebe5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.534320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04f71b-ea58-489b-b8b3-b5f0a40c7ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.538917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04f71b-ea58-489b-b8b3-b5f0a40c7ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.539944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04f71b-ea58-489b-b8b3-b5f0a40c7ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.544254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04f71b-ea58-489b-b8b3-b5f0a40c7ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.545643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04f71b-ea58-489b-b8b3-b5f0a40c7ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.547617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04f71b-ea58-489b-b8b3-b5f0a40c7ce2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.565097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1029/2031] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:48.569998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.589683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff34805-c694-4f6e-9011-641e270d145f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.591052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.591355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.591639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.592010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] PATH_INFO: `/v3/projects/98f9ed539ac04bdea587e1e1ebadb53e/users/64b7f81be8c74c86a8b19822fc9710ac/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.616230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f9ed539ac04bdea587e1e1ebadb53e, user_id=64b7f81be8c74c86a8b19822fc9710ac, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.622427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.669707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1003/2032] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:48 2022] PUT /identity/v3/projects/98f9ed539ac04bdea587e1e1ebadb53e/users/64b7f81be8c74c86a8b19822fc9710ac/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:48.675924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b27df6-c022-46d8-b97f-f5dd982ddeee None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.676140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b27df6-c022-46d8-b97f-f5dd982ddeee None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.676581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b27df6-c022-46d8-b97f-f5dd982ddeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.742458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-13b27df6-c022-46d8-b97f-f5dd982ddeee None None] MFA Rules not processed for user `64b7f81be8c74c86a8b19822fc9710ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:48.748297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.750829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:48.752166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-13b27df6-c022-46d8-b97f-f5dd982ddeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:48.887001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8bd54-5c99-4f78-8c08-c2f9a133be88 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.897227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.898641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.900473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.901171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] PATH_INFO: `/v3/projects/44a68b82c03649a58b251f892243c270` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.904084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a68b82c03649a58b251f892243c270)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.906398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1030/2033] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:48.911753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.913635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b27df6-c022-46d8-b97f-f5dd982ddeee None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:48.935416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:48.935989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:48.944748 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b27df6-c022-46d8-b97f-f5dd982ddeee None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.952103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.952821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.953295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.953798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.954711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.957049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:49.003936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a68b82c03649a58b251f892243c270', 'request_id': 'req-559637e1-0a60-4c07-8a87-5ef3e7263fd9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.008957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1031/2034] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:48 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:49.010801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:49.010921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a68b82c03649a58b251f892243c270', 'request_id': 'req-559637e1-0a60-4c07-8a87-5ef3e7263fd9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.019520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a68b82c03649a58b251f892243c270', 'request_id': 'req-559637e1-0a60-4c07-8a87-5ef3e7263fd9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.052261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] The token cache is being invalidate because project 44a68b82c03649a58b251f892243c270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:49.052591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.053491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.055577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1004/2035] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:48 2022] DELETE /identity/v3/projects/44a68b82c03649a58b251f892243c270 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:49.060787 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:49.063426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:49.140285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a5b86-aa05-49ea-bc5f-4fe043b18b9b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:49.146165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:49.146901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.147272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.147848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.148580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:49.151233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:49.181229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1032/2036] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:49 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:49.185406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:49.187491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:49.205994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8359-b453-4cf8-bd9e-0a8ea53332c0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:49.208108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559637e1-0a60-4c07-8a87-5ef3e7263fd9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:49.212058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:49.212826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.213114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.213391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.214079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:49.214881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:49.215655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.215875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:49.216000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.216203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] PATH_INFO: `/v3/users/4291202c7c8a4c61b31db9c4e4ed0bfe` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.218886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4291202c7c8a4c61b31db9c4e4ed0bfe)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:49.221760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:49.229051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1033/2037] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:49.233869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:49.244897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8013f-e9b1-409e-90e1-cd8ce66e55dd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:49.251858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:49.253031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.253717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.254636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] PATH_INFO: `/v3/projects/3b4b92aed4684d0a8f893dcf854df1f7/users/a2f1c5d0c2ec4c9382bcc4341a37bbd5/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.277932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4b92aed4684d0a8f893dcf854df1f7, user_id=a2f1c5d0c2ec4c9382bcc4341a37bbd5, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:49.283178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:49.305753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1034/2038] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:49 2022] PUT /identity/v3/projects/3b4b92aed4684d0a8f893dcf854df1f7/users/a2f1c5d0c2ec4c9382bcc4341a37bbd5/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:49.311497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:49.333532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4291202c7c8a4c61b31db9c4e4ed0bfe'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.349267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4291202c7c8a4c61b31db9c4e4ed0bfe'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.385958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4291202c7c8a4c61b31db9c4e4ed0bfe'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.387383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4291202c7c8a4c61b31db9c4e4ed0bfe'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.412984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4291202c7c8a4c61b31db9c4e4ed0bfe'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:49.433817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1005/2039] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:49 2022] DELETE /identity/v3/users/4291202c7c8a4c61b31db9c4e4ed0bfe => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:49.442775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd73950-1e8f-4f53-a443-dc2de3d3b503 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:49.535371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:49.576364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4e789-a81c-402f-b131-23635c133f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:49.577622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e789-a81c-402f-b131-23635c133f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.578086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e789-a81c-402f-b131-23635c133f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.579371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e789-a81c-402f-b131-23635c133f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.580686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4e789-a81c-402f-b131-23635c133f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:49.584835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4e789-a81c-402f-b131-23635c133f24 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:49.632285 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:49.634236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:49.641115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1035/2040] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:49.660531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9198-1d75-4b58-8aa9-4e4dec8c46cc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:49.666969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:49.667678 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.667962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.668237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] PATH_INFO: `/v3/projects/3b4b92aed4684d0a8f893dcf854df1f7/users/a2f1c5d0c2ec4c9382bcc4341a37bbd5/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.679358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4b92aed4684d0a8f893dcf854df1f7, user_id=a2f1c5d0c2ec4c9382bcc4341a37bbd5, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:49.683605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:49.721637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1006/2041] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:49 2022] PUT /identity/v3/projects/3b4b92aed4684d0a8f893dcf854df1f7/users/a2f1c5d0c2ec4c9382bcc4341a37bbd5/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:49.731465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.732102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.732450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.781396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:49.783854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:49.804590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:49.821112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] MFA Rules not processed for user `a2f1c5d0c2ec4c9382bcc4341a37bbd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:49.823230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:49.912715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94f2fd-fe90-48a2-a615-0935dca2b670 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:49.921685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:49.922542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:49.922926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:49.923550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] PATH_INFO: `/v3/users/601a92ab01324b53b9952df4d225cc4b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:49.924940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] RBAC: Authorizing `identity:delete_user(user_id=601a92ab01324b53b9952df4d225cc4b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:49.927928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:50.024104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1036/2042] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:50.025896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:50.028138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.086130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601a92ab01324b53b9952df4d225cc4b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.119612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601a92ab01324b53b9952df4d225cc4b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.170264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601a92ab01324b53b9952df4d225cc4b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.177386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601a92ab01324b53b9952df4d225cc4b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.208926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed57163-e7f0-468c-9243-22541c4f47a3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:50.212270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ed498b73-1e01-408f-beaf-3b1fec3a616e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601a92ab01324b53b9952df4d225cc4b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.217865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:50.220380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.220852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.221241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.226998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.242983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1007/2043] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:07:49 2022] DELETE /identity/v3/users/601a92ab01324b53b9952df4d225cc4b => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:50.250724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.254922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.255591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.317694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:50.320948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.407113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:50.409180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:50.430726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1037/2044] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:50.465543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1008/2045] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:50.472480 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:50.476792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.500397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:50.502558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.508150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:50.527706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ead79d-e9a8-40c3-b4e9-d6628dab3a9c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:50.535578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:50.537984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.538335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.538685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] PATH_INFO: `/v3/projects/b8e6562d60fd4eab875a848cb89fb941` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.540920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e6562d60fd4eab875a848cb89fb941)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:50.543841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:50.580058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625515f-9cd4-4666-a06f-22e0af67407b service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:50.608946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:50.666360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e6562d60fd4eab875a848cb89fb941', 'request_id': 'req-de22a0f9-2885-4b21-a6af-33f2bbac0f51'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.679321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e6562d60fd4eab875a848cb89fb941', 'request_id': 'req-de22a0f9-2885-4b21-a6af-33f2bbac0f51'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.691321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e6562d60fd4eab875a848cb89fb941', 'request_id': 'req-de22a0f9-2885-4b21-a6af-33f2bbac0f51'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.693543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff754d-c8c2-4249-9f25-c8f82e45b2be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:50.694231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff754d-c8c2-4249-9f25-c8f82e45b2be service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.694502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff754d-c8c2-4249-9f25-c8f82e45b2be service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.694763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff754d-c8c2-4249-9f25-c8f82e45b2be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.712136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] The token cache is being invalidate because project b8e6562d60fd4eab875a848cb89fb941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:50.712474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.714073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-de22a0f9-2885-4b21-a6af-33f2bbac0f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:50.718045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1009/2046] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:50 2022] DELETE /identity/v3/projects/b8e6562d60fd4eab875a848cb89fb941 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:50.721696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff754d-c8c2-4249-9f25-c8f82e45b2be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:50.723804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff754d-c8c2-4249-9f25-c8f82e45b2be service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:50.724521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed96458-ae2e-485c-a752-1b20811e5395 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.725853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed96458-ae2e-485c-a752-1b20811e5395 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.727684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed96458-ae2e-485c-a752-1b20811e5395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.741655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1038/2047] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:50.775800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4ed96458-ae2e-485c-a752-1b20811e5395 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:50.779195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ed96458-ae2e-485c-a752-1b20811e5395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.965907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1010/2048] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:50.973106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53360-4376-4a13-a656-8211d33d25ff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.973662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53360-4376-4a13-a656-8211d33d25ff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.974005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53360-4376-4a13-a656-8211d33d25ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.007237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e3f53360-4376-4a13-a656-8211d33d25ff None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:51.009341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e3f53360-4376-4a13-a656-8211d33d25ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:51.023411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1039/2049] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:51.028802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed96458-ae2e-485c-a752-1b20811e5395 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.045967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed96458-ae2e-485c-a752-1b20811e5395 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.055059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.056102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.056576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.057632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.058601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:51.059072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.063432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.083641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1011/2050] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:51.088156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f53360-4376-4a13-a656-8211d33d25ff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.099031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f53360-4376-4a13-a656-8211d33d25ff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.104286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.105038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.105381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.105745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.106455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.108512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.174101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1040/2051] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:51 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:51.178864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.190261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3675627-3d0d-430a-af48-41176e16f44d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.307223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.310145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.310609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.311093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.311980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.314589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.341473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1012/2052] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:51 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:51.345847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.357808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fe4be-b179-4a5e-900b-05a501489edf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.364013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.364934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.365630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.365630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.366357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.368094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.390976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1041/2053] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:51.397482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.429299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319de1-1182-4b38-965d-115cb79a32a0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.435344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3400fe-50cf-439b-9cc1-15565752a6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.435972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3400fe-50cf-439b-9cc1-15565752a6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.436255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3400fe-50cf-439b-9cc1-15565752a6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.436541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3400fe-50cf-439b-9cc1-15565752a6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.437195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3400fe-50cf-439b-9cc1-15565752a6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.439062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3400fe-50cf-439b-9cc1-15565752a6b1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.476666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1013/2054] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:51.478924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.499582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea237-bae2-490c-b797-e9dce79ae8f9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.509888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.511834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.512527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.512991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] PATH_INFO: `/v3/projects/a2d1aeb2eb2b497b854d6c052059222c/users/86dece6b122b4677a321d8e74ff80424/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.546130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d1aeb2eb2b497b854d6c052059222c, user_id=86dece6b122b4677a321d8e74ff80424, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.549266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.571726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1042/2055] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:51 2022] PUT /identity/v3/projects/a2d1aeb2eb2b497b854d6c052059222c/users/86dece6b122b4677a321d8e74ff80424/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:51.593458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.594654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.595665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.667815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:51.680078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] MFA Rules not processed for user `86dece6b122b4677a321d8e74ff80424`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:51.685490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:51.805125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1014/2056] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:51.819429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.821194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:51.926448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3133a0b-569f-4951-8a0d-016c973c0e78 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.935797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241ac8a-c9d3-49c3-b9d3-b1c41563835b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.936610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ac8a-c9d3-49c3-b9d3-b1c41563835b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.936954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ac8a-c9d3-49c3-b9d3-b1c41563835b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.937709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ac8a-c9d3-49c3-b9d3-b1c41563835b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.950300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241ac8a-c9d3-49c3-b9d3-b1c41563835b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.952082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241ac8a-c9d3-49c3-b9d3-b1c41563835b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.956361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.957869 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:51.964035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1043/2057] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:52.156225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a70038-31d8-4385-a9ba-3ac261bd73ba None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.166459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.167292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.167679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.168064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] PATH_INFO: `/v3/projects/b2d8c64afff440919c4e5b6cd66494de` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.173366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d8c64afff440919c4e5b6cd66494de)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.176299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.292770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d8c64afff440919c4e5b6cd66494de', 'request_id': 'req-f5186312-abfb-4870-9439-e573b199b851'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:52.299700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d8c64afff440919c4e5b6cd66494de', 'request_id': 'req-f5186312-abfb-4870-9439-e573b199b851'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:52.310792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d8c64afff440919c4e5b6cd66494de', 'request_id': 'req-f5186312-abfb-4870-9439-e573b199b851'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:52.339240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] The token cache is being invalidate because project b2d8c64afff440919c4e5b6cd66494de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:07:52.340254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:52.341849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5186312-abfb-4870-9439-e573b199b851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:07:52.345264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1015/2058] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:07:51 2022] DELETE /identity/v3/projects/b2d8c64afff440919c4e5b6cd66494de => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:52.351998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777135-2c93-471e-bdcb-85a522e1cfdc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.352517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777135-2c93-471e-bdcb-85a522e1cfdc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.352856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777135-2c93-471e-bdcb-85a522e1cfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.396200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-75777135-2c93-471e-bdcb-85a522e1cfdc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:52.398166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-75777135-2c93-471e-bdcb-85a522e1cfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:52.422960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:52.544051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1044/2059] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:52.552932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cca9cd-f4f5-46d9-be7e-a5dbde4b76c4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.553363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cca9cd-f4f5-46d9-be7e-a5dbde4b76c4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.554306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cca9cd-f4f5-46d9-be7e-a5dbde4b76c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.578325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d7cca9cd-f4f5-46d9-be7e-a5dbde4b76c4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:52.579618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d7cca9cd-f4f5-46d9-be7e-a5dbde4b76c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:52.593913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75777135-2c93-471e-bdcb-85a522e1cfdc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.604477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1016/2060] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:52.605692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75777135-2c93-471e-bdcb-85a522e1cfdc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.610930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.611556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.611835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.622647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.623465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:52.623785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.629982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.659958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cca9cd-f4f5-46d9-be7e-a5dbde4b76c4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.668836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1045/2061] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:52.683537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cca9cd-f4f5-46d9-be7e-a5dbde4b76c4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.699958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.700994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.701439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.701986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.702845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.705918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.745259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1017/2062] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:07:52 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:52.749540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.760810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0dd60-e969-446e-bff0-56dc967f1924 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.829490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.830232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.830604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.830931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.831672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.833969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.862116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1046/2063] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:07:52 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:52.863939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.873804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:52.879124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef4dd5-8b47-4b2f-a2d7-712b4d5a378b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.888620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.889580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.890162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.890682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.891483 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.893562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.911423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1018/2064] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:52.915978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.934500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101803c-e045-4d6f-a728-03c5354f8700 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.942359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d7bb1-9121-4023-9aae-de7a429d78ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.943227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d7bb1-9121-4023-9aae-de7a429d78ee admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.943581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d7bb1-9121-4023-9aae-de7a429d78ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.944867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d7bb1-9121-4023-9aae-de7a429d78ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.945884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d7bb1-9121-4023-9aae-de7a429d78ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.948135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d7bb1-9121-4023-9aae-de7a429d78ee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.961260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1047/2065] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:07:52.967188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.980345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd18555-0eae-4e85-8803-34434dba335e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.992949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.993955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.994272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.994652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] PATH_INFO: `/v3/projects/4da3acfcc34b44a5909e036705c040c8/users/86032a9fd3b8429b99a7d23ddaf03116/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.014019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da3acfcc34b44a5909e036705c040c8, user_id=86032a9fd3b8429b99a7d23ddaf03116, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:53.017659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:53.034706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1019/2066] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:07:52 2022] PUT /identity/v3/projects/4da3acfcc34b44a5909e036705c040c8/users/86032a9fd3b8429b99a7d23ddaf03116/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:53.040554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f97db-9fea-4272-9c36-5b2366be9623 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.040878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f97db-9fea-4272-9c36-5b2366be9623 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.041117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f97db-9fea-4272-9c36-5b2366be9623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.064097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-785f97db-9fea-4272-9c36-5b2366be9623 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:53.065510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-785f97db-9fea-4272-9c36-5b2366be9623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:53.160391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1048/2067] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:53.179636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:53.182036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:53.290236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a2f81-af93-4e86-b9f6-74578cac4a74 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:53.294155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:53.294792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.295076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.295396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.305973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:53.307444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:53.319348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1020/2068] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:54.392785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.393770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.393770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.434366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] MFA Rules not processed for user `86032a9fd3b8429b99a7d23ddaf03116`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:54.435477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:54.452316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:54.535515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1049/2069] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:54.548086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:54.549786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:54.658545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1344c2-9bd1-43d2-866a-0097584319f0 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:54.663428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a223886-041b-4262-aa9f-7217ddccbf7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:54.664374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a223886-041b-4262-aa9f-7217ddccbf7a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.665039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a223886-041b-4262-aa9f-7217ddccbf7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.665491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a223886-041b-4262-aa9f-7217ddccbf7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.679426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a223886-041b-4262-aa9f-7217ddccbf7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:54.681467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a223886-041b-4262-aa9f-7217ddccbf7a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:54.702045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1021/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:57.748869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:57.751231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:57.766152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ca2e8-1a79-4702-9d5e-c5e58b1320ed None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.771032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.771893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.772180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.772441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.774604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:57.895419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:57.897802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:57.917814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1050/2071] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:59.336398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fd51b-5d25-4867-9d27-96485375a97c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.338104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fd51b-5d25-4867-9d27-96485375a97c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.338464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fd51b-5d25-4867-9d27-96485375a97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.369347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7d7fd51b-5d25-4867-9d27-96485375a97c None None] MFA Rules not processed for user `86032a9fd3b8429b99a7d23ddaf03116`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:59.371028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7d7fd51b-5d25-4867-9d27-96485375a97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:59.381750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1022/2072] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:07:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:05.872389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:05.875133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:05.893287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a40a-0b10-4811-b6d0-12cdb37c5d16 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:05.907622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:05.908580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:05.908988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:05.909392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] PATH_INFO: `/v3/users/86dece6b122b4677a321d8e74ff80424` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:05.911840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86dece6b122b4677a321d8e74ff80424)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:05.915129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:06.018647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86dece6b122b4677a321d8e74ff80424'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:06.051967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86dece6b122b4677a321d8e74ff80424'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:06.070995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86dece6b122b4677a321d8e74ff80424'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:06.072338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86dece6b122b4677a321d8e74ff80424'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:06.091684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86dece6b122b4677a321d8e74ff80424'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:06.105550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1051/2073] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:05 2022] DELETE /identity/v3/users/86dece6b122b4677a321d8e74ff80424 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:06.112480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:06.113067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:06.113524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:06.174009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:06.174009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:06.323823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1023/2074] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:06.354298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:06.360157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:06.467436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:06.605465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202b239-8b43-4fb4-8a81-eb588d08525b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:06.611887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:06.613212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:06.613869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:06.614284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:06.637148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:06.640266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:06.661262 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1052/2075] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:07.139922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:07.141981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:07.161429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d917b63-a75a-436b-a56e-6d3348b8d55c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:07.168359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:07.169183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:07.169559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:07.169988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] PATH_INFO: `/v3/projects/a2d1aeb2eb2b497b854d6c052059222c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:07.171507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d1aeb2eb2b497b854d6c052059222c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:07.174304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:07.177805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:07.280295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d1aeb2eb2b497b854d6c052059222c', 'request_id': 'req-adadbd72-ff6e-4df6-9f64-50d98847dbf5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:07.290875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d1aeb2eb2b497b854d6c052059222c', 'request_id': 'req-adadbd72-ff6e-4df6-9f64-50d98847dbf5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:07.308084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d1aeb2eb2b497b854d6c052059222c', 'request_id': 'req-adadbd72-ff6e-4df6-9f64-50d98847dbf5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:07.336242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] The token cache is being invalidate because project a2d1aeb2eb2b497b854d6c052059222c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:08:07.336677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:07.338273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adadbd72-ff6e-4df6-9f64-50d98847dbf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:07.347547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1024/2076] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:08:07 2022] DELETE /identity/v3/projects/a2d1aeb2eb2b497b854d6c052059222c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:16.649950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=17 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:17.340433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:37.459515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.461383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:37.476148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bed9f-83b9-4482-b1ab-90aba83a5670 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:37.574978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:37.575851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.576322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.582308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] PATH_INFO: `/v3/users/64b7f81be8c74c86a8b19822fc9710ac` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:37.584479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b7f81be8c74c86a8b19822fc9710ac)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:37.586632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:37.657369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b7f81be8c74c86a8b19822fc9710ac'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:37.682606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b7f81be8c74c86a8b19822fc9710ac'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:37.696021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b7f81be8c74c86a8b19822fc9710ac'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:37.697145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b7f81be8c74c86a8b19822fc9710ac'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:37.710395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b7f81be8c74c86a8b19822fc9710ac'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:37.719218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1053/2077] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:37 2022] DELETE /identity/v3/users/64b7f81be8c74c86a8b19822fc9710ac => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:37.725490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.726513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.726513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:37.751219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:37.752507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:37.882158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1025/2078] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:37.897575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.900409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:37.990258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f38df-991a-4e7c-8135-8a9198f6bea3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:37.994596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:37.995241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.995534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.995790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:38.011523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:38.013638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:38.043994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1054/2079] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:38.867694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:38.872962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:38.904552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47a2ef-8b71-4dd9-9265-2aec7255a42d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:38.915956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:38.917762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:38.918569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:38.919242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] PATH_INFO: `/v3/projects/98f9ed539ac04bdea587e1e1ebadb53e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:38.922874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f9ed539ac04bdea587e1e1ebadb53e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:38.927951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:39.104527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f9ed539ac04bdea587e1e1ebadb53e', 'request_id': 'req-a36298a5-c6ff-43b8-8f2a-95633aed33c3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.113565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f9ed539ac04bdea587e1e1ebadb53e', 'request_id': 'req-a36298a5-c6ff-43b8-8f2a-95633aed33c3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.125060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f9ed539ac04bdea587e1e1ebadb53e', 'request_id': 'req-a36298a5-c6ff-43b8-8f2a-95633aed33c3'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.146002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] The token cache is being invalidate because project 98f9ed539ac04bdea587e1e1ebadb53e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:08:39.146621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.147549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.154168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1026/2080] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:08:38 2022] DELETE /identity/v3/projects/98f9ed539ac04bdea587e1e1ebadb53e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:39.156777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:39.180523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a8147-3e7e-4b2b-b736-32fda0483d3c service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:39.242096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:39.367385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:39.370511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:39.370967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:39.371622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] PATH_INFO: `/v3/users/a2f1c5d0c2ec4c9382bcc4341a37bbd5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:39.386882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f1c5d0c2ec4c9382bcc4341a37bbd5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:39.390778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:39.524410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f1c5d0c2ec4c9382bcc4341a37bbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.545549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f1c5d0c2ec4c9382bcc4341a37bbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.559745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f1c5d0c2ec4c9382bcc4341a37bbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.560469 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f1c5d0c2ec4c9382bcc4341a37bbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.572751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-83a5ce3f-0687-4071-b4aa-e2018f5ef35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f1c5d0c2ec4c9382bcc4341a37bbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:39.581695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1055/2081] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:39 2022] DELETE /identity/v3/users/a2f1c5d0c2ec4c9382bcc4341a37bbd5 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:40.603121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:40.605974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:40.633627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36298a5-c6ff-43b8-8f2a-95633aed33c3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:40.730454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:40.731897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:40.732401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:40.732883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] PATH_INFO: `/v3/projects/3b4b92aed4684d0a8f893dcf854df1f7` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:40.735115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4b92aed4684d0a8f893dcf854df1f7)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:40.739344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:40.802426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:40.849731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4b92aed4684d0a8f893dcf854df1f7', 'request_id': 'req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:40.859068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4b92aed4684d0a8f893dcf854df1f7', 'request_id': 'req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:40.867808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4b92aed4684d0a8f893dcf854df1f7', 'request_id': 'req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:40.897784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] The token cache is being invalidate because project 3b4b92aed4684d0a8f893dcf854df1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:08:40.898285 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:40.899746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e8ec3d56-068a-4de6-9cb1-69e95b3a0c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:40.902785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1027/2082] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:08:40 2022] DELETE /identity/v3/projects/3b4b92aed4684d0a8f893dcf854df1f7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:40.910891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cdfb-6fd2-4095-ab19-0aed4f4494ea None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:40.912082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cdfb-6fd2-4095-ab19-0aed4f4494ea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:40.912082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cdfb-6fd2-4095-ab19-0aed4f4494ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:40.946797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7b09cdfb-6fd2-4095-ab19-0aed4f4494ea None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:40.949219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7b09cdfb-6fd2-4095-ab19-0aed4f4494ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:41.062409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1056/2083] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:41.072376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9698-29ca-49d2-b407-97183a5e37bd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.072878 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9698-29ca-49d2-b407-97183a5e37bd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.073364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9698-29ca-49d2-b407-97183a5e37bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.110358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2f2e9698-29ca-49d2-b407-97183a5e37bd None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:41.112913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2f2e9698-29ca-49d2-b407-97183a5e37bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:41.130216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1028/2084] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:41.136588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09cdfb-6fd2-4095-ab19-0aed4f4494ea None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.158671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09cdfb-6fd2-4095-ab19-0aed4f4494ea None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.167303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.168514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.169083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.169618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.170610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:41.171149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.173350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:41.209924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1057/2085] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:41.215681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9698-29ca-49d2-b407-97183a5e37bd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.229362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9698-29ca-49d2-b407-97183a5e37bd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.237210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.238261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.238747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.239368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.240617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.244628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:41.295674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1029/2086] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:08:41 2022] POST /identity/v3/projects => generated 401 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:41.302573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.337032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4ff00-f76f-4bfa-a90f-01e23a851ccf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.372197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:41.422064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.422713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.423003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.423529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.424361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.427142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:41.465725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1058/2087] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:08:41 2022] POST /identity/v3/users => generated 301 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:41.469124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.485863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe50609-1406-4f66-927e-81ecdc723a61 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.493082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.493745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.494037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.494362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.496542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.497011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:41.518884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1030/2088] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:08:41.525988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.539663 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa45f4-2e72-4855-a45b-55b10551ca53 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.547795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498a766-bb27-4480-9f9d-18191444875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.548643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a766-bb27-4480-9f9d-18191444875e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.549058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a766-bb27-4480-9f9d-18191444875e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.549532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a766-bb27-4480-9f9d-18191444875e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.550563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498a766-bb27-4480-9f9d-18191444875e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.552612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498a766-bb27-4480-9f9d-18191444875e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:41.569746 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1059/2089] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:08:41.575941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.594720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39d73-7f02-4774-a291-b5409df0924d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.604276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.605621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.606275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.606890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] PATH_INFO: `/v3/projects/6e342f507cd54bb999ea2f783fa9ad45/users/6760a8b62be44bc8ac47c8ed03647c8f/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.636006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e342f507cd54bb999ea2f783fa9ad45, user_id=6760a8b62be44bc8ac47c8ed03647c8f, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.641564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:41.666642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1031/2090] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:41 2022] PUT /identity/v3/projects/6e342f507cd54bb999ea2f783fa9ad45/users/6760a8b62be44bc8ac47c8ed03647c8f/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:41.678865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ae52-9b9e-49c5-9b89-1b334dc2e126 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.679157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ae52-9b9e-49c5-9b89-1b334dc2e126 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.679414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ae52-9b9e-49c5-9b89-1b334dc2e126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.720798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8089ae52-9b9e-49c5-9b89-1b334dc2e126 None None] MFA Rules not processed for user `6760a8b62be44bc8ac47c8ed03647c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:41.722156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8089ae52-9b9e-49c5-9b89-1b334dc2e126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:41.834171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1060/2091] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:41 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:41.839977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.976406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ed111-4742-4e4b-9bfb-63a9cacff357 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.983934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.984594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.985988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.986303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.987440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.990690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.031967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1032/2092] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:08:41 2022] POST /identity/v3/projects => generated 399 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:42.043607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089ae52-9b9e-49c5-9b89-1b334dc2e126 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.052408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089ae52-9b9e-49c5-9b89-1b334dc2e126 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.151458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.152454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.152847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.153193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.153960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.156550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.193781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1061/2093] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:08:42 2022] POST /identity/v3/users => generated 306 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:08:42.198305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.223806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604264f-d0d1-48fd-923b-685e3a559167 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.235331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.236338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.236819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.237480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.238518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.242905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.267394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1033/2094] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:08:42.277465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.302605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa24a1-72c3-40df-9a32-f79e240765f7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.308348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.309030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.309304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.309572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] PATH_INFO: `/v3/projects/f14de61bc497430d87a1e2491d9e6825/users/7fdca21162424bd69569db75e3aed166/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.339709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14de61bc497430d87a1e2491d9e6825, user_id=7fdca21162424bd69569db75e3aed166, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.345432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.368776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1062/2095] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:42 2022] PUT /identity/v3/projects/f14de61bc497430d87a1e2491d9e6825/users/7fdca21162424bd69569db75e3aed166/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:42.372941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.390384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc9cc1-2ac9-4695-9ddb-753cff68f882 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.429890 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:42.502268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ba0fe-ecf9-43a8-8d0f-ff7c30b0a487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.503322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ba0fe-ecf9-43a8-8d0f-ff7c30b0a487 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.503770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ba0fe-ecf9-43a8-8d0f-ff7c30b0a487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.504264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ba0fe-ecf9-43a8-8d0f-ff7c30b0a487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.505494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ba0fe-ecf9-43a8-8d0f-ff7c30b0a487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.508599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ba0fe-ecf9-43a8-8d0f-ff7c30b0a487 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.529114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1034/2096] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:08:42.540319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.546539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:42.558535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056ac32-5b3b-40d6-87be-df9570406057 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.568619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.569653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.570121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.570536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] PATH_INFO: `/v3/projects/f14de61bc497430d87a1e2491d9e6825/users/7fdca21162424bd69569db75e3aed166/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.575113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14de61bc497430d87a1e2491d9e6825, user_id=7fdca21162424bd69569db75e3aed166, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.578518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.592120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1063/2097] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:42 2022] PUT /identity/v3/projects/f14de61bc497430d87a1e2491d9e6825/users/7fdca21162424bd69569db75e3aed166/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:42.612368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033550ee-c41d-4304-9b11-3e46400cd22f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.613916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033550ee-c41d-4304-9b11-3e46400cd22f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.619162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033550ee-c41d-4304-9b11-3e46400cd22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.682966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-033550ee-c41d-4304-9b11-3e46400cd22f None None] MFA Rules not processed for user `7fdca21162424bd69569db75e3aed166`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:42.684511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-033550ee-c41d-4304-9b11-3e46400cd22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:42.838215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1035/2098] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:42 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:42.850349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.853807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:42.965312 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50586a2-cd99-4071-8eb1-a0c8e3aede57 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.972587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a67eae-aa38-4fb6-961b-e3cfb08ff8c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.974282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a67eae-aa38-4fb6-961b-e3cfb08ff8c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.975013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a67eae-aa38-4fb6-961b-e3cfb08ff8c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.975692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a67eae-aa38-4fb6-961b-e3cfb08ff8c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.995250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a67eae-aa38-4fb6-961b-e3cfb08ff8c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.998334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a67eae-aa38-4fb6-961b-e3cfb08ff8c6 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:43.019169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1064/2099] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:42 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:43.197280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033550ee-c41d-4304-9b11-3e46400cd22f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:43.215856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033550ee-c41d-4304-9b11-3e46400cd22f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:43.222189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:43.223158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:43.223707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:43.224288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:43.372265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:43.376924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:43.430653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1036/2100] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:43 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:46.325184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.326025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.326025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.373630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:46.384857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:46.491992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1065/2101] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:08:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:46.549436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.551065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:46.627821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:46.677962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4dabb-c042-4b7a-8430-eff2efc324cb service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.688205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ddd82-f3c7-4a33-9172-f1677972547b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.689350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ddd82-f3c7-4a33-9172-f1677972547b service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.690296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ddd82-f3c7-4a33-9172-f1677972547b service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.690792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ddd82-f3c7-4a33-9172-f1677972547b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.709309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ddd82-f3c7-4a33-9172-f1677972547b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:46.712255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ddd82-f3c7-4a33-9172-f1677972547b service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:46.731624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1037/2102] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:08:47.477945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:47.480310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:47.691960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359977c3-db93-4a37-b0d0-f712ed0bcff2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:47.701155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:47.702169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.702537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.702958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] PATH_INFO: `/v3/users/c0db39216fd24b5f9cb0531bcedb0326` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.704589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0db39216fd24b5f9cb0531bcedb0326)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:47.707132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:47.776854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:47.790249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0db39216fd24b5f9cb0531bcedb0326'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:47.815053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0db39216fd24b5f9cb0531bcedb0326'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:47.827381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0db39216fd24b5f9cb0531bcedb0326'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:47.828091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0db39216fd24b5f9cb0531bcedb0326'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:47.838783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0db39216fd24b5f9cb0531bcedb0326'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:47.846261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1066/2103] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:47 2022] DELETE /identity/v3/users/c0db39216fd24b5f9cb0531bcedb0326 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:47.951568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.952440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.952440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.986118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:47.988414 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.092226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.094090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.111210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1038/2104] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 13:08:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:48.149211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.152042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.202584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25afbc-a6f5-474e-b127-78d642faff40 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:48.208709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:48.209582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.209969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.210308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] PATH_INFO: `/v3/projects/9a847ae68ca242c0bce510fd91c2633c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.212338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a847ae68ca242c0bce510fd91c2633c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:48.214678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:48.250703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbd90f-8383-4f1a-b5b8-cf0e2c7f2331 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:48.257560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fe288-b1a3-434b-af6a-32588725c284 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:48.258407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe288-b1a3-434b-af6a-32588725c284 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.258624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe288-b1a3-434b-af6a-32588725c284 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.259000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fe288-b1a3-434b-af6a-32588725c284 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.269805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fe288-b1a3-434b-af6a-32588725c284 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:48.271874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fe288-b1a3-434b-af6a-32588725c284 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:48.282695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1039/2105] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:08:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:08:48.289244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a847ae68ca242c0bce510fd91c2633c', 'request_id': 'req-69a8e0de-45ab-4817-af67-a5f040c9d34f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:48.298554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a847ae68ca242c0bce510fd91c2633c', 'request_id': 'req-69a8e0de-45ab-4817-af67-a5f040c9d34f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:48.312071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a847ae68ca242c0bce510fd91c2633c', 'request_id': 'req-69a8e0de-45ab-4817-af67-a5f040c9d34f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:48.328526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] The token cache is being invalidate because project 9a847ae68ca242c0bce510fd91c2633c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:08:48.328830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:48.329929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69a8e0de-45ab-4817-af67-a5f040c9d34f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:48.331549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1067/2106] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:08:48 2022] DELETE /identity/v3/projects/9a847ae68ca242c0bce510fd91c2633c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:08:48.339227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d08-8b5f-42bb-906e-891e59ed74a9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.339602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d08-8b5f-42bb-906e-891e59ed74a9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.340099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d08-8b5f-42bb-906e-891e59ed74a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.381817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ae864d08-8b5f-42bb-906e-891e59ed74a9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:48.383797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ae864d08-8b5f-42bb-906e-891e59ed74a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.499802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1040/2107] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:48.509622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2a0c6-99ed-4606-8e8b-e2470b7c4aa4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.511228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2a0c6-99ed-4606-8e8b-e2470b7c4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.511981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2a0c6-99ed-4606-8e8b-e2470b7c4aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.542465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8ea2a0c6-99ed-4606-8e8b-e2470b7c4aa4 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:48.544466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8ea2a0c6-99ed-4606-8e8b-e2470b7c4aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.556059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1068/2108] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:48.562438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae864d08-8b5f-42bb-906e-891e59ed74a9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.574188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae864d08-8b5f-42bb-906e-891e59ed74a9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:48.583720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5154ca-d7ef-48fe-9281-befa4ca3ef70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:48.585008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5154ca-d7ef-48fe-9281-befa4ca3ef70 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.585596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5154ca-d7ef-48fe-9281-befa4ca3ef70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.586170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5154ca-d7ef-48fe-9281-befa4ca3ef70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.587579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5154ca-d7ef-48fe-9281-befa4ca3ef70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:48.588027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5154ca-d7ef-48fe-9281-befa4ca3ef70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:48.591957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5154ca-d7ef-48fe-9281-befa4ca3ef70 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:48.596012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:48.629441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1041/2109] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:48.631601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2a0c6-99ed-4606-8e8b-e2470b7c4aa4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.655176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2a0c6-99ed-4606-8e8b-e2470b7c4aa4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:48.666089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:48.667117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.667881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.668579 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.669971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:48.674166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:48.746841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.747464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.748002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.755052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1069/2110] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:08:48 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:08:48.761081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.803888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:48.806574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.807426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:48.924885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd96a-5226-49f4-80af-c088d3534a61 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:48.934292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:48.935528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.936003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.936559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.938084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:48.942059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:48.947762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1042/2111] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:48.972889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.976475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.980133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1070/2112] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:08:48 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:48.986945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:49.005533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e2736-82ee-4a62-900b-ffc424d1feae admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.014043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.015190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.015888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.016058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.017086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.020549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.041481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1071/2113] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:08:49.047022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:49.067682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3daab0-78da-4f20-bae6-0da795d71a23 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.076170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.077013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.077385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.077848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.078641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.081454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.098437 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1072/2114] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:08:49.100185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:49.103896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:49.108525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c035ce-109b-4fb1-815d-41c5f8e2334d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.115395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.116326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.116750 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.117115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.122150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f504cf-b68f-46a1-a117-1417ccc97e76 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.129456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.130949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.131301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.131707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] PATH_INFO: `/v3/users/6760a8b62be44bc8ac47c8ed03647c8f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.131969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.133860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] RBAC: Authorizing `identity:delete_user(user_id=6760a8b62be44bc8ac47c8ed03647c8f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.134927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.137555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.153784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1043/2115] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:08:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:08:49.154958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:49.174929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c07a4-be97-44de-8cb6-485787803107 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.190466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4d387-314e-46a7-a584-60898f657607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.191770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4d387-314e-46a7-a584-60898f657607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.192148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4d387-314e-46a7-a584-60898f657607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.192652 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4d387-314e-46a7-a584-60898f657607 admin admin] PATH_INFO: `/v3/projects/b33fcaf2f89d4438947508dee3737cee/users/30153f2f99424f29afaf718784c9f6b1/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.226425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a4d387-314e-46a7-a584-60898f657607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33fcaf2f89d4438947508dee3737cee, user_id=30153f2f99424f29afaf718784c9f6b1, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.230229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a4d387-314e-46a7-a584-60898f657607 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.232230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6760a8b62be44bc8ac47c8ed03647c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:49.250840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1044/2116] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:49 2022] PUT /identity/v3/projects/b33fcaf2f89d4438947508dee3737cee/users/30153f2f99424f29afaf718784c9f6b1/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:49.251318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6760a8b62be44bc8ac47c8ed03647c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:49.256632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.256915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.257215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.265807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6760a8b62be44bc8ac47c8ed03647c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:49.266848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6760a8b62be44bc8ac47c8ed03647c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:49.282573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:49.282744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:49.290568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0c379b23-32d8-41aa-8277-35ccb7862cee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6760a8b62be44bc8ac47c8ed03647c8f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:49.304571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1073/2117] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:49 2022] DELETE /identity/v3/users/6760a8b62be44bc8ac47c8ed03647c8f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:49.314833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.315488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.315850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.343787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:49.345551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:49.393543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1045/2118] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:49.413449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:49.416833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:49.436583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:49.465738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:49.485858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1074/2119] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:49.502216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:49.505870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:49.564126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12aee7-32e9-4139-ad35-44a9f641e741 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.664913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c5fee-4efb-4218-b3b5-c14e1087dffc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.672695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.673914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.674388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.674853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.679152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c92be4-b39b-47c0-8f88-48996bf449b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.680421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c92be4-b39b-47c0-8f88-48996bf449b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.680421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c92be4-b39b-47c0-8f88-48996bf449b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.680829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c92be4-b39b-47c0-8f88-48996bf449b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.694553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c92be4-b39b-47c0-8f88-48996bf449b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.694957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.696623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c92be4-b39b-47c0-8f88-48996bf449b4 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.698043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.711367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1046/2120] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:08:49.726620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1075/2121] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:08:49.788835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.789167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.789376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.850946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:49.852232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:50.121227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1047/2122] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 13:08:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:50.152777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.156667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:50.421094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d5bd0-34cb-43be-9da4-8e1aaed4f4c3 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:50.430232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:50.430560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.431037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.431466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.446945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:50.450323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:50.483926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1076/2123] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:08:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:08:50.595570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.604086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:50.643007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069445a-cbe0-4e1c-804c-b3a1e371e647 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:50.658066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:50.661247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.675919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.677260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] PATH_INFO: `/v3/projects/6e342f507cd54bb999ea2f783fa9ad45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.680668 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e342f507cd54bb999ea2f783fa9ad45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:50.685825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:50.800830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e342f507cd54bb999ea2f783fa9ad45', 'request_id': 'req-14642b62-5f68-443d-aece-766e3e28bc90'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:50.828457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e342f507cd54bb999ea2f783fa9ad45', 'request_id': 'req-14642b62-5f68-443d-aece-766e3e28bc90'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:50.842356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e342f507cd54bb999ea2f783fa9ad45', 'request_id': 'req-14642b62-5f68-443d-aece-766e3e28bc90'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:50.848123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:50.876513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] The token cache is being invalidate because project 6e342f507cd54bb999ea2f783fa9ad45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:08:50.877077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:50.879366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:50.884261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1048/2124] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:08:50 2022] DELETE /identity/v3/projects/6e342f507cd54bb999ea2f783fa9ad45 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:08:50.889331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.909435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662174a-aec8-4179-96f1-4e2c1270f97c service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:51.027249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:51.028179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:51.028650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:51.029344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] PATH_INFO: `/v3/users/7fdca21162424bd69569db75e3aed166` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:51.031396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fdca21162424bd69569db75e3aed166)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:51.033834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:51.106055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:51.127295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fdca21162424bd69569db75e3aed166'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:51.149443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fdca21162424bd69569db75e3aed166'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:51.163646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fdca21162424bd69569db75e3aed166'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:51.164864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fdca21162424bd69569db75e3aed166'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:51.175281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1bba4140-5c84-48c1-b8de-813a0d7392c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fdca21162424bd69569db75e3aed166'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:51.187868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1077/2125] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:50 2022] DELETE /identity/v3/users/7fdca21162424bd69569db75e3aed166 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:52.011261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.015326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:52.053060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642b62-5f68-443d-aece-766e3e28bc90 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.211198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:52.212908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.212908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.213307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] PATH_INFO: `/v3/projects/f14de61bc497430d87a1e2491d9e6825` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.215663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14de61bc497430d87a1e2491d9e6825)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:52.219663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:52.341460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14de61bc497430d87a1e2491d9e6825', 'request_id': 'req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:52.358343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14de61bc497430d87a1e2491d9e6825', 'request_id': 'req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:52.374969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14de61bc497430d87a1e2491d9e6825', 'request_id': 'req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:52.421158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] The token cache is being invalidate because project f14de61bc497430d87a1e2491d9e6825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:08:52.421885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:52.424471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d49c5fdd-4626-4bb8-9617-1ec76aa954ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:52.429744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1049/2126] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:08:52 2022] DELETE /identity/v3/projects/f14de61bc497430d87a1e2491d9e6825 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:52.436954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b965aa-858e-418b-931e-f25cdc03538a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.437447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b965aa-858e-418b-931e-f25cdc03538a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.438079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b965aa-858e-418b-931e-f25cdc03538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.491424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-46b965aa-858e-418b-931e-f25cdc03538a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:52.494198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-46b965aa-858e-418b-931e-f25cdc03538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:52.629032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1078/2127] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:52.642233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ac272-3af1-4fc3-b190-1cf1c24cf078 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.642680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ac272-3af1-4fc3-b190-1cf1c24cf078 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.643074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ac272-3af1-4fc3-b190-1cf1c24cf078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.698981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6a1ac272-3af1-4fc3-b190-1cf1c24cf078 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:52.702020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6a1ac272-3af1-4fc3-b190-1cf1c24cf078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:52.718225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1050/2128] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:52.724853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b965aa-858e-418b-931e-f25cdc03538a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.737772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b965aa-858e-418b-931e-f25cdc03538a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.750824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:52.751990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.752674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.753239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.754680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:52.755355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:52.758838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:52.784707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1079/2129] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:52.793421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ac272-3af1-4fc3-b190-1cf1c24cf078 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.815609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ac272-3af1-4fc3-b190-1cf1c24cf078 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.821924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:52.822687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.823028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.823304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.824054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:52.827733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:52.874674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1051/2130] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:08:52 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:52.876410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.891758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb8e37-b331-4d84-a2ef-2573a0c144e0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.014329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.015815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.016347 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.016717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.017504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.020226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.056438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1080/2131] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:08:52 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:08:53.058783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.071799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcf00-25c4-42ea-a46e-66188e5235ca admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.077977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.078614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.078930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.079201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.079877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.082083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.105534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1052/2132] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:08:53.106984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.124402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b340ea5-8599-4ad6-adba-fc4eb28491cd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.131979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c274403-dbc7-464a-8d07-2966af1a6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.132876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274403-dbc7-464a-8d07-2966af1a6d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.135205 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274403-dbc7-464a-8d07-2966af1a6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.136002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274403-dbc7-464a-8d07-2966af1a6d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.139550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c274403-dbc7-464a-8d07-2966af1a6d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.143164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c274403-dbc7-464a-8d07-2966af1a6d27 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.197719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1081/2133] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:08:53.203926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.226644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef02f61-60d8-465f-81f1-888f7db79c63 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.245062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.247348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.247826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.248144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] PATH_INFO: `/v3/projects/0a2629a8d3a44bd18bf4898813d6976f/users/7ce330bca8d54401961875dcd69578a4/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.294426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2629a8d3a44bd18bf4898813d6976f, user_id=7ce330bca8d54401961875dcd69578a4, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.299052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.323887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1053/2134] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:53 2022] PUT /identity/v3/projects/0a2629a8d3a44bd18bf4898813d6976f/users/7ce330bca8d54401961875dcd69578a4/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:53.353216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.353976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.355085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.475356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] MFA Rules not processed for user `7ce330bca8d54401961875dcd69578a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:53.476692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:53.483357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.587496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1082/2135] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:53.594696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.626952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.745353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f29c7c-7745-440b-a499-17232ad5f73e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.790351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.791189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.791790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.792306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.793513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.796085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.814556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.817734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:53.863461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1054/2136] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:08:53 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:08:53.870006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f2e51-a979-4f99-90ec-a2a09ef40cba None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.871502 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.095880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f0b6-7c4b-4488-8986-0502d77bab40 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.110552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.111224 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.111511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.111785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] PATH_INFO: `/v3/users/c589c9ed14594c498eb5a2af90030231` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.112963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.113940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.114782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.115288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.116272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.116484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c589c9ed14594c498eb5a2af90030231)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.119266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.119866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.191777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1055/2137] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:08:53 2022] POST /identity/v3/users => generated 298 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:54.196720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.235985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69d40-0666-4529-9c15-a0fbc1c8111f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.260503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.261841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.262992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.263915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.265434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c589c9ed14594c498eb5a2af90030231'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:54.265886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.269221 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.294207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c589c9ed14594c498eb5a2af90030231'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:54.299680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1056/2138] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:08:54.307157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.311376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c589c9ed14594c498eb5a2af90030231'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:54.312407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c589c9ed14594c498eb5a2af90030231'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:54.331977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-afbc9472-c09a-4349-8bb8-0c0a026ad17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c589c9ed14594c498eb5a2af90030231'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:54.347667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1083/2139] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:53 2022] DELETE /identity/v3/users/c589c9ed14594c498eb5a2af90030231 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:54.355698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.356242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.356686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.357712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6871b-0295-4438-a6e5-179755211738 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.444317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:54.452907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:54.487333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.488191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.488483 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.501698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] PATH_INFO: `/v3/projects/2b9640dac5b646e58edde23ecefbb889/users/2721b80d4ba945d6b1b32fe1484119e8/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.508994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.534614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b9640dac5b646e58edde23ecefbb889, user_id=2721b80d4ba945d6b1b32fe1484119e8, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.540948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.584182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1057/2140] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:54 2022] PUT /identity/v3/projects/2b9640dac5b646e58edde23ecefbb889/users/2721b80d4ba945d6b1b32fe1484119e8/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:08:54.595140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.601021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1084/2141] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:54.603493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:54.622228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.640997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.643290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:54.768398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c03626-cf72-4008-b81f-fcdf919b3bfe admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.778568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.779991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.780478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.780957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.782191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.784118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa4749-18af-48d4-8cec-985fc579d360 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.785496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.810546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1058/2142] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:08:54.816359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.840300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48279e7-0a8f-433c-86eb-042d5edd6797 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.852415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae1ea3-7145-4cec-b7ed-0be2abd86691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.853127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae1ea3-7145-4cec-b7ed-0be2abd86691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.853435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae1ea3-7145-4cec-b7ed-0be2abd86691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.853788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae1ea3-7145-4cec-b7ed-0be2abd86691 admin admin] PATH_INFO: `/v3/projects/2b9640dac5b646e58edde23ecefbb889/users/2721b80d4ba945d6b1b32fe1484119e8/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.856328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae1ea3-7145-4cec-b7ed-0be2abd86691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b9640dac5b646e58edde23ecefbb889, user_id=2721b80d4ba945d6b1b32fe1484119e8, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.860857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae1ea3-7145-4cec-b7ed-0be2abd86691 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.874189 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1059/2143] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:54 2022] PUT /identity/v3/projects/2b9640dac5b646e58edde23ecefbb889/users/2721b80d4ba945d6b1b32fe1484119e8/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:54.894593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.895568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.896439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.916348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.917030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.918147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.918429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.946326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.947853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.967125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1085/2144] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:08:54.972024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] MFA Rules not processed for user `2721b80d4ba945d6b1b32fe1484119e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:54.973286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:55.111778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1060/2145] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:55.131699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:55.134390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:55.297536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:55.299971 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:55.360832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf44a-2744-4c56-95ad-7735b62c9b64 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.535381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:55.550474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2348e-930a-4453-b9ef-73bf6a01cbd9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.551767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.552811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.553994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.554306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.558755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.559865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.560353 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.560842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] PATH_INFO: `/v3/projects/1f2e1474e0024ac3918f295c3e228869` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.563180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2e1474e0024ac3918f295c3e228869)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.566766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.567146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.569702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.580521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1086/2146] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:55.619257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:55.727943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2e1474e0024ac3918f295c3e228869', 'request_id': 'req-118b9ea5-c84e-4918-973d-897e94a0179f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:55.754120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:55.770793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2e1474e0024ac3918f295c3e228869', 'request_id': 'req-118b9ea5-c84e-4918-973d-897e94a0179f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:55.778018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f21c-8d6b-4dcd-90a0-e768712ecb21 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.791661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2e1474e0024ac3918f295c3e228869', 'request_id': 'req-118b9ea5-c84e-4918-973d-897e94a0179f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:55.845939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] The token cache is being invalidate because project 1f2e1474e0024ac3918f295c3e228869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:08:55.846528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:55.847845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-118b9ea5-c84e-4918-973d-897e94a0179f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:08:55.857820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1061/2147] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:08:55 2022] DELETE /identity/v3/projects/1f2e1474e0024ac3918f295c3e228869 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:08:55.863926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac50692-0688-458a-8ceb-a96897809c4a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.864860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac50692-0688-458a-8ceb-a96897809c4a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.864860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac50692-0688-458a-8ceb-a96897809c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.931046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cac50692-0688-458a-8ceb-a96897809c4a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:55.932131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cac50692-0688-458a-8ceb-a96897809c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:55.957917 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.962685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.964534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.964919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.976298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:56.116916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1062/2148] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:56.130055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27911f9-7ce9-4d32-b6ef-09c6f2c492da None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.131227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27911f9-7ce9-4d32-b6ef-09c6f2c492da None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.131625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27911f9-7ce9-4d32-b6ef-09c6f2c492da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.158532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:56.160033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:56.181865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1087/2149] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:08:56.193533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d27911f9-7ce9-4d32-b6ef-09c6f2c492da None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:56.197820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d27911f9-7ce9-4d32-b6ef-09c6f2c492da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:56.239162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1063/2150] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:56.246732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:56.269867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3fe9e-35eb-4139-acd6-fb4eedd95514 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:56.283337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:56.285156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.286556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.287531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.289109 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:56.290113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:56.297928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:56.336733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1088/2151] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:56.342906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27911f9-7ce9-4d32-b6ef-09c6f2c492da None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:56.361950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27911f9-7ce9-4d32-b6ef-09c6f2c492da None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:56.372576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:56.373784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.374347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.374844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.376194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:56.380348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:56.456904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1064/2152] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:08:56 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:08:56.459397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:56.489142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7c645-9563-4148-8080-96d5ba069b13 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:56.692203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:56.693772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.694557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.695241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.696261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:56.700595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:56.734735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1089/2153] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:08:56 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:08:56.740760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:56.761404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29456a5-3d4d-4936-9a78-7c7546952164 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:56.776510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:56.778553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.778961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.779539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.781078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:56.784763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:56.812941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1065/2154] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:08:56.814905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:56.840212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d8062-d9c6-4ab3-8597-87a794f3a58e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:56.851596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f89a6-977c-47b9-ad7a-36d50e0e9663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:56.853406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f89a6-977c-47b9-ad7a-36d50e0e9663 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.854134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f89a6-977c-47b9-ad7a-36d50e0e9663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.854693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f89a6-977c-47b9-ad7a-36d50e0e9663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.856053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f89a6-977c-47b9-ad7a-36d50e0e9663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:56.861280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f89a6-977c-47b9-ad7a-36d50e0e9663 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:56.894680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1090/2155] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:08:56.901210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:56.936480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9316e95-22b2-463b-bdc6-0abfca621c5b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:56.952312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:56.953903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.954510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.955884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] PATH_INFO: `/v3/projects/73afbf716079452ba1deb906dbdce949/users/b22dd3b0461747688993851ccf4e66d0/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:57.019582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73afbf716079452ba1deb906dbdce949, user_id=b22dd3b0461747688993851ccf4e66d0, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:57.032418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:57.086609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1066/2156] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:08:56 2022] PUT /identity/v3/projects/73afbf716079452ba1deb906dbdce949/users/b22dd3b0461747688993851ccf4e66d0/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:57.089643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5328d88-92b7-487c-828e-c9ac7c804cfa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:57.090298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5328d88-92b7-487c-828e-c9ac7c804cfa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:57.090739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5328d88-92b7-487c-828e-c9ac7c804cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:57.130407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:57.149578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f5328d88-92b7-487c-828e-c9ac7c804cfa None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:57.150968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f5328d88-92b7-487c-828e-c9ac7c804cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:57.277842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1091/2157] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:57.287786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:57.291177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:57.418981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:57.438496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0400f-fcf5-4156-b225-caa99333d202 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:57.451236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb38d7c-d35a-4978-a765-c048c566ac6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:57.452082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38d7c-d35a-4978-a765-c048c566ac6f service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:57.452656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38d7c-d35a-4978-a765-c048c566ac6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:57.452955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb38d7c-d35a-4978-a765-c048c566ac6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:57.471529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb38d7c-d35a-4978-a765-c048c566ac6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:57.473421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb38d7c-d35a-4978-a765-c048c566ac6f service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:57.500880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1067/2158] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:59.088132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:59.089410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:59.089887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:59.179473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] MFA Rules not processed for user `30153f2f99424f29afaf718784c9f6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:59.187990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:59.321835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:59.322313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:59.322636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:59.387992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1092/2159] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:59.397660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:59.400317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:59.433337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] MFA Rules not processed for user `b22dd3b0461747688993851ccf4e66d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:59.435921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:59.530842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d5546-7ac1-4953-a2a6-8c578526fbff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:59.546446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:59.547591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:59.548047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:59.548315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:59.587028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:59.591017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:59.636552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1093/2160] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:59.649896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1068/2161] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:08:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:59.659773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:59.662965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:59.698110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:59.703556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:59.768633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac71f5-e101-4fc2-820a-81a606f18cab None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:59.773907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:59.785793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:59.789280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:59.790076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:59.790331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] PATH_INFO: `/v3/users/7ce330bca8d54401961875dcd69578a4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:59.807187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce330bca8d54401961875dcd69578a4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:59.814574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:59.906157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abf05b-af01-4880-96b8-f69167ba5c9a service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:59.917889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23aaee-fd00-4dfd-94ca-12383ee992a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:59.921945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23aaee-fd00-4dfd-94ca-12383ee992a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:59.922360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23aaee-fd00-4dfd-94ca-12383ee992a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:59.922650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23aaee-fd00-4dfd-94ca-12383ee992a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:59.947962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23aaee-fd00-4dfd-94ca-12383ee992a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:59.949500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23aaee-fd00-4dfd-94ca-12383ee992a2 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:59.982989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1094/2162] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:00.009571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce330bca8d54401961875dcd69578a4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:00.034986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce330bca8d54401961875dcd69578a4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:00.065934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce330bca8d54401961875dcd69578a4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:00.066984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce330bca8d54401961875dcd69578a4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:00.081718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce330bca8d54401961875dcd69578a4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:00.093156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1069/2163] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:08:59 2022] DELETE /identity/v3/users/7ce330bca8d54401961875dcd69578a4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:00.099368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338858-83e2-4d8a-be74-5dbbab0c47c7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.100653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338858-83e2-4d8a-be74-5dbbab0c47c7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.101405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338858-83e2-4d8a-be74-5dbbab0c47c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.132993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-58338858-83e2-4d8a-be74-5dbbab0c47c7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:00.134863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-58338858-83e2-4d8a-be74-5dbbab0c47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:00.354801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1095/2164] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:00.400421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:00.403242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:00.561988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:00.579549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2c73b-73fa-411c-b1e2-9c7d66ab4b6a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:00.598303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:00.600727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.600727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.601245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.647352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:00.650222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:00.682532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1070/2165] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:09:02.215034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7018cd1-0e8b-4738-9cf7-85b729288b21 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.216528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7018cd1-0e8b-4738-9cf7-85b729288b21 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.217249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7018cd1-0e8b-4738-9cf7-85b729288b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.228160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.230394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:02.279928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd9861-b466-4eb4-add2-96ca05f62e41 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.300681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.301113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b7018cd1-0e8b-4738-9cf7-85b729288b21 None None] MFA Rules not processed for user `b22dd3b0461747688993851ccf4e66d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:02.301615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.302196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.302756 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] PATH_INFO: `/v3/projects/0a2629a8d3a44bd18bf4898813d6976f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.305893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b7018cd1-0e8b-4738-9cf7-85b729288b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:02.311160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2629a8d3a44bd18bf4898813d6976f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.314555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.445055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1096/2166] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:02.479017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2629a8d3a44bd18bf4898813d6976f', 'request_id': 'req-edfe6804-cd95-452a-b362-b982294ab6c8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.497454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2629a8d3a44bd18bf4898813d6976f', 'request_id': 'req-edfe6804-cd95-452a-b362-b982294ab6c8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.510282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2629a8d3a44bd18bf4898813d6976f', 'request_id': 'req-edfe6804-cd95-452a-b362-b982294ab6c8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.539201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] The token cache is being invalidate because project 0a2629a8d3a44bd18bf4898813d6976f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:09:02.539615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.540832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edfe6804-cd95-452a-b362-b982294ab6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.548282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1071/2167] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:09:02 2022] DELETE /identity/v3/projects/0a2629a8d3a44bd18bf4898813d6976f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:02.549444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7018cd1-0e8b-4738-9cf7-85b729288b21 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.576333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7018cd1-0e8b-4738-9cf7-85b729288b21 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.708916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.709596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.709916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.710218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] PATH_INFO: `/v3/users/2721b80d4ba945d6b1b32fe1484119e8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.711590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2721b80d4ba945d6b1b32fe1484119e8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.714202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.725314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:02.833978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2721b80d4ba945d6b1b32fe1484119e8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.880614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2721b80d4ba945d6b1b32fe1484119e8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.893052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2721b80d4ba945d6b1b32fe1484119e8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.894067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2721b80d4ba945d6b1b32fe1484119e8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.905527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2721b80d4ba945d6b1b32fe1484119e8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:02.914152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1097/2168] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:09:02 2022] DELETE /identity/v3/users/2721b80d4ba945d6b1b32fe1484119e8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:03.728130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03410ed7-c8a5-4e30-a9c4-d90a2b0a6493 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:03.729035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03410ed7-c8a5-4e30-a9c4-d90a2b0a6493 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:03.731899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03410ed7-c8a5-4e30-a9c4-d90a2b0a6493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:03.799430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-03410ed7-c8a5-4e30-a9c4-d90a2b0a6493 None None] MFA Rules not processed for user `30153f2f99424f29afaf718784c9f6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:03.801405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-03410ed7-c8a5-4e30-a9c4-d90a2b0a6493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:03.980760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:03.988081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:04.034884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1072/2169] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:04.233012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1272bf-8728-4f13-a95d-b3c65065b331 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:04.257826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:04.277735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.277735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.277735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] PATH_INFO: `/v3/projects/2b9640dac5b646e58edde23ecefbb889` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.277735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b9640dac5b646e58edde23ecefbb889)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:04.277735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:04.435350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b9640dac5b646e58edde23ecefbb889', 'request_id': 'req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:04.446922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b9640dac5b646e58edde23ecefbb889', 'request_id': 'req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:04.465065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b9640dac5b646e58edde23ecefbb889', 'request_id': 'req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:04.493815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] The token cache is being invalidate because project 2b9640dac5b646e58edde23ecefbb889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:09:04.494279 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:04.495479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20b2c1dd-c0b3-435c-ab0f-9b11be3cbbd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:04.507359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1098/2170] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:09:03 2022] DELETE /identity/v3/projects/2b9640dac5b646e58edde23ecefbb889 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:04.508235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9052-b11c-40c6-a32c-e375b4a7a7c5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.508671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9052-b11c-40c6-a32c-e375b4a7a7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.509087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9052-b11c-40c6-a32c-e375b4a7a7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.551993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-560a9052-b11c-40c6-a32c-e375b4a7a7c5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:04.553896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-560a9052-b11c-40c6-a32c-e375b4a7a7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:04.676389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1073/2171] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:04.680759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9294236-13b8-4209-99d0-b3784b007464 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.682083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9294236-13b8-4209-99d0-b3784b007464 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.682620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9294236-13b8-4209-99d0-b3784b007464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.711679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a9294236-13b8-4209-99d0-b3784b007464 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:04.713968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a9294236-13b8-4209-99d0-b3784b007464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:04.726578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1099/2172] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:04.734614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a9052-b11c-40c6-a32c-e375b4a7a7c5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:04.763636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a9052-b11c-40c6-a32c-e375b4a7a7c5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:04.774956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:04.776572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.777588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.778185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.779032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:04.779481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:04.781491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:04.808484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1074/2173] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:04.814474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9294236-13b8-4209-99d0-b3784b007464 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:04.835837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9294236-13b8-4209-99d0-b3784b007464 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:04.863679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:04.864306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.866431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.866980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.868967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:04.874540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:04.885404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:04.929304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1100/2174] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:09:04 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:04.938386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:04.952297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:04.962042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318c18-bb92-497d-9a36-769d329843d8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:05.192396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:05.193690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.193934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.194228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.194960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:05.197287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:05.251489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1075/2175] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:09:04 2022] POST /identity/v3/users => generated 287 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:05.260356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:05.302272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179bfd-992f-4678-837a-8ed3a077da8c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:05.334925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:05.336495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.337645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.338964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.340683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:05.344216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:05.375720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1101/2176] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:05.377380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:05.417545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ddaa2-5bb1-473a-845e-a0e6ab24e802 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:05.427484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12eac2-d53c-43be-9f17-c2a689b117aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:05.428895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12eac2-d53c-43be-9f17-c2a689b117aa admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.429755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12eac2-d53c-43be-9f17-c2a689b117aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.430736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12eac2-d53c-43be-9f17-c2a689b117aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.432709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12eac2-d53c-43be-9f17-c2a689b117aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:05.443365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12eac2-d53c-43be-9f17-c2a689b117aa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:05.473741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1076/2177] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:05.479086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:05.499782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58843797-a40a-4b8f-a8d5-e134ad7bc840 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:05.518002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:05.529174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.545006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.548601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] PATH_INFO: `/v3/projects/83a783cf223b44a0b78156198bb3a469/users/86db15481e044bcea0e19d2dbfd244b6/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.647436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a783cf223b44a0b78156198bb3a469, user_id=86db15481e044bcea0e19d2dbfd244b6, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:05.652984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:05.702624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1102/2178] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:09:05 2022] PUT /identity/v3/projects/83a783cf223b44a0b78156198bb3a469/users/86db15481e044bcea0e19d2dbfd244b6/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:05.708477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c5d86-9fc1-4bac-9f03-f2d76cf98c9b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.708941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c5d86-9fc1-4bac-9f03-f2d76cf98c9b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.709400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c5d86-9fc1-4bac-9f03-f2d76cf98c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.871207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0f8c5d86-9fc1-4bac-9f03-f2d76cf98c9b None None] MFA Rules not processed for user `86db15481e044bcea0e19d2dbfd244b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:05.880610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0f8c5d86-9fc1-4bac-9f03-f2d76cf98c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:06.189592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1077/2179] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:06.225646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:06.228464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:06.423809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3e2ba-1da6-4bf8-8cdd-66291e9dfb48 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:06.434428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d9846-af60-498d-8eb3-ef9441d0d41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:06.435668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d9846-af60-498d-8eb3-ef9441d0d41b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:06.435959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d9846-af60-498d-8eb3-ef9441d0d41b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:06.436472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d9846-af60-498d-8eb3-ef9441d0d41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:06.465263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d9846-af60-498d-8eb3-ef9441d0d41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:06.470492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d9846-af60-498d-8eb3-ef9441d0d41b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:06.487488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1103/2180] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:09:06.514014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c5d86-9fc1-4bac-9f03-f2d76cf98c9b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:06.625271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c5d86-9fc1-4bac-9f03-f2d76cf98c9b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:06.630695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:06.631436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:06.631740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:06.632083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] PATH_INFO: `/v3/users/86db15481e044bcea0e19d2dbfd244b6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:06.633396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=86db15481e044bcea0e19d2dbfd244b6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:06.635504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:06.655400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:06.703979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86db15481e044bcea0e19d2dbfd244b6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:06.716497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86db15481e044bcea0e19d2dbfd244b6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:06.735198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86db15481e044bcea0e19d2dbfd244b6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:06.736768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86db15481e044bcea0e19d2dbfd244b6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:06.749842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29cace14-c1a1-4131-b4ba-980cffb18f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86db15481e044bcea0e19d2dbfd244b6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:06.758125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1078/2181] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:09:06 2022] DELETE /identity/v3/users/86db15481e044bcea0e19d2dbfd244b6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:06.765380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:06.765380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:06.765380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:06.789002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:06.789334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:06.789662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:06.796585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:06.798354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:06.816950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] MFA Rules not processed for user `86032a9fd3b8429b99a7d23ddaf03116`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:06.818491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:06.880875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1104/2182] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:06.893334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1079/2183] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:06.912087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:06.916031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:07.087247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644eb72-94b4-4269-a61e-8198bddbad9a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:07.105240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f7577-b14d-4294-882d-21e45b96d21e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:07.107359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7577-b14d-4294-882d-21e45b96d21e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:07.110050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7577-b14d-4294-882d-21e45b96d21e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:07.111221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7577-b14d-4294-882d-21e45b96d21e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:07.145566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f7577-b14d-4294-882d-21e45b96d21e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:07.151415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f7577-b14d-4294-882d-21e45b96d21e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:07.176857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:07.202125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1105/2184] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:08.692365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:08.698361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:08.775976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147e54c-1912-4ce9-a0ff-8fa813a09f52 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:08.798134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:08.799641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:08.800438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:08.800881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] PATH_INFO: `/v3/projects/83a783cf223b44a0b78156198bb3a469` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:08.803459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a783cf223b44a0b78156198bb3a469)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:08.807536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:08.916320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a783cf223b44a0b78156198bb3a469', 'request_id': 'req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:08.933164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a783cf223b44a0b78156198bb3a469', 'request_id': 'req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:08.948362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a783cf223b44a0b78156198bb3a469', 'request_id': 'req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:09.007475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] The token cache is being invalidate because project 83a783cf223b44a0b78156198bb3a469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:09:09.008481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:09.010163 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-61bcf0cd-4c5b-4a1e-be94-a9d0e68af811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:09.019781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1080/2185] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:09:08 2022] DELETE /identity/v3/projects/83a783cf223b44a0b78156198bb3a469 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:09.027276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc1fd-d5a7-45a8-ad0f-2eb84ba0d13c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.030828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc1fd-d5a7-45a8-ad0f-2eb84ba0d13c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.031739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc1fd-d5a7-45a8-ad0f-2eb84ba0d13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.079297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-62afc1fd-d5a7-45a8-ad0f-2eb84ba0d13c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:09.082351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-62afc1fd-d5a7-45a8-ad0f-2eb84ba0d13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:09.246710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1106/2186] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:09.266934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834bb3c-c65f-4a63-a090-cefcb11a1ea8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.267252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834bb3c-c65f-4a63-a090-cefcb11a1ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.267517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834bb3c-c65f-4a63-a090-cefcb11a1ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.326155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3834bb3c-c65f-4a63-a090-cefcb11a1ea8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:09.328215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3834bb3c-c65f-4a63-a090-cefcb11a1ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:09.343712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1081/2187] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:09.350487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc1fd-d5a7-45a8-ad0f-2eb84ba0d13c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.367204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc1fd-d5a7-45a8-ad0f-2eb84ba0d13c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.391471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.392875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.393588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.394498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.396501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:09.397825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:09.401752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:09.446199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1107/2188] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:09.453172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834bb3c-c65f-4a63-a090-cefcb11a1ea8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.493740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834bb3c-c65f-4a63-a090-cefcb11a1ea8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.504234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.505842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.506366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.506837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.516034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:09.527047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:09.570357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:09.615085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1082/2189] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:09:09 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:09.621539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.640412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58801e1-37b1-4b4c-bf8d-58a34a965d05 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.798513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.799767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.801786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.802603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.804046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:09.819616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:09.926190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1108/2190] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:09:09 2022] POST /identity/v3/users => generated 289 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:09:09.927077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.963761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604d8cc-82dd-46cb-9cc4-9aff55f7587e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.971091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.972282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.973621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.974428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.975262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:09.978067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.026080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1083/2191] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:09:10.028399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.051220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a55bb-ef90-465c-8769-b10ec18d8a9e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.076577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef95d9b-6fa0-43eb-9277-9611f2d184c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.078375 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95d9b-6fa0-43eb-9277-9611f2d184c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.079767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95d9b-6fa0-43eb-9277-9611f2d184c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.081028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95d9b-6fa0-43eb-9277-9611f2d184c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.083997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef95d9b-6fa0-43eb-9277-9611f2d184c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.088555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef95d9b-6fa0-43eb-9277-9611f2d184c8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.134786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1109/2192] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:10.142696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.190223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bde7eb-a0e2-4196-ad3e-faf85fc10be5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.245500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.247293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.248036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.249497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] PATH_INFO: `/v3/projects/413f90a33acf4f768282cf2288795728/users/ce01aa1b29dd41fcab68765154234e6c/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.297059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=413f90a33acf4f768282cf2288795728, user_id=ce01aa1b29dd41fcab68765154234e6c, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.304378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.342351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1084/2193] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:09:10 2022] PUT /identity/v3/projects/413f90a33acf4f768282cf2288795728/users/ce01aa1b29dd41fcab68765154234e6c/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:10.353617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652578-ef62-430c-b6d4-5809ed16419a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.353938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652578-ef62-430c-b6d4-5809ed16419a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.354200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652578-ef62-430c-b6d4-5809ed16419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.426103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-16652578-ef62-430c-b6d4-5809ed16419a None None] MFA Rules not processed for user `ce01aa1b29dd41fcab68765154234e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:10.429559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-16652578-ef62-430c-b6d4-5809ed16419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:10.579573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:10.616161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1110/2194] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:10.622545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.754172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abb2a3-9a42-45f9-907a-af3f88460c3c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.765147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.765873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.766473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.767130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.768497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.773746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.828906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1085/2195] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:09:10 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:10.835930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652578-ef62-430c-b6d4-5809ed16419a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.858526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652578-ef62-430c-b6d4-5809ed16419a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.110274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.113588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.114164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.114748 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.116072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.122002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.197054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.200596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1111/2196] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:09:10 2022] POST /identity/v3/users => generated 288 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:11.242247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae19dc-3f19-47c1-82fd-f5897dde28c7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.269941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.273354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.276893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.277694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.288806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.292871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.324177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1086/2197] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:11.334951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.372858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736c543-c158-4fd0-8f37-c16e7788ee2d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.396244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae5a2a-6131-471c-b7ef-70396af54455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.397937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae5a2a-6131-471c-b7ef-70396af54455 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.399032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae5a2a-6131-471c-b7ef-70396af54455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.399793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae5a2a-6131-471c-b7ef-70396af54455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.403194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae5a2a-6131-471c-b7ef-70396af54455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.406379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae5a2a-6131-471c-b7ef-70396af54455 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.450573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1112/2198] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:11.453473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.515445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f28f-ceca-4d24-a61f-b41abcf7e641 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.531846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.535231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.539203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.545427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] PATH_INFO: `/v3/projects/513f1afb1b05478e8b27af897703e429/users/e0eb300e662d4908bb23689e407667c6/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.622704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=513f1afb1b05478e8b27af897703e429, user_id=e0eb300e662d4908bb23689e407667c6, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.628230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.673284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1087/2199] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:09:11 2022] PUT /identity/v3/projects/513f1afb1b05478e8b27af897703e429/users/e0eb300e662d4908bb23689e407667c6/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:11.694628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69f84c-9ffa-499c-8c81-8a866614d902 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.695150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69f84c-9ffa-499c-8c81-8a866614d902 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.695632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69f84c-9ffa-499c-8c81-8a866614d902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.759403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8e69f84c-9ffa-499c-8c81-8a866614d902 None None] MFA Rules not processed for user `e0eb300e662d4908bb23689e407667c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:11.763364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e69f84c-9ffa-499c-8c81-8a866614d902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:11.920599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1113/2200] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:11.927204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.070753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2139c11-03dc-41ec-8c01-8bde6d024bea admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.086151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.087752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.088088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.089261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.089261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.095087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.115618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:12.146302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1088/2201] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:09:11 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:12.151948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69f84c-9ffa-499c-8c81-8a866614d902 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.171665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69f84c-9ffa-499c-8c81-8a866614d902 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.317989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.319688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.320406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.320950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.322958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.326462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.344631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:12.422956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1114/2202] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:09:12 2022] POST /identity/v3/users => generated 295 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:12.428939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.451558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1884c4-c2c0-4c05-9a29-2db2b3983089 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.462985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.464319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.464975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.465892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.467993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.472513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.500702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1089/2203] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:12.505055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.518759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7310-9d88-4b77-ac4c-c91a0796bb84 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.534058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7657f7f-ab22-4616-9c4e-e9e0616c30c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.534853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7657f7f-ab22-4616-9c4e-e9e0616c30c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.536056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7657f7f-ab22-4616-9c4e-e9e0616c30c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.536557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7657f7f-ab22-4616-9c4e-e9e0616c30c0 admin admin] PATH_INFO: `/v3/projects/6987bab936d444aa815a8bf7571355c0/users/101a8fb5057d466fb0097c0f2539664b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.563956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7657f7f-ab22-4616-9c4e-e9e0616c30c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6987bab936d444aa815a8bf7571355c0, user_id=101a8fb5057d466fb0097c0f2539664b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.566969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7657f7f-ab22-4616-9c4e-e9e0616c30c0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.600250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1115/2204] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:09:12 2022] PUT /identity/v3/projects/6987bab936d444aa815a8bf7571355c0/users/101a8fb5057d466fb0097c0f2539664b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:12.605023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.610055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42334c1-004a-451d-bb13-a7c5ec475d58 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.611627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42334c1-004a-451d-bb13-a7c5ec475d58 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.612173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42334c1-004a-451d-bb13-a7c5ec475d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.625273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863bd8e-8d6f-427c-ba81-fe8f013460a8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.651992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d42334c1-004a-451d-bb13-a7c5ec475d58 None None] MFA Rules not processed for user `3cad6d025da54eb29d09fd2a2888c844`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:12.654440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d42334c1-004a-451d-bb13-a7c5ec475d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:12.728303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1116/2205] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:12.791579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612549f-16f4-4433-b861-75fdd75d9af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.792840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612549f-16f4-4433-b861-75fdd75d9af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.793482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612549f-16f4-4433-b861-75fdd75d9af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.794142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612549f-16f4-4433-b861-75fdd75d9af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.795556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612549f-16f4-4433-b861-75fdd75d9af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.799035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612549f-16f4-4433-b861-75fdd75d9af5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.821745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1090/2206] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:12.825464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42334c1-004a-451d-bb13-a7c5ec475d58 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.847210 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42334c1-004a-451d-bb13-a7c5ec475d58 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.856489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.857642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.858272 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.858863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] PATH_INFO: `/v3/projects/6987bab936d444aa815a8bf7571355c0/users/101a8fb5057d466fb0097c0f2539664b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.862640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6987bab936d444aa815a8bf7571355c0, user_id=101a8fb5057d466fb0097c0f2539664b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.868050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.892908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1117/2207] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:09:12 2022] PUT /identity/v3/projects/6987bab936d444aa815a8bf7571355c0/users/101a8fb5057d466fb0097c0f2539664b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:12.913802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.914326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.914873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.977243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] MFA Rules not processed for user `101a8fb5057d466fb0097c0f2539664b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:12.978082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:13.155973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1091/2208] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:13.171177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.174125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:13.283975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183969a-ab72-43e9-b2a8-c46f2a4fb018 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.292325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.293793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.294798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.295353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.449963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:13.453129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:13.481074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1118/2209] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:15.685826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:15.688687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:15.718552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d9dd-f23c-424d-9e80-fa86fe54ee27 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:15.726166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:15.734296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.735522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.735851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.883744 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:15.887300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:15.899254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:15.908886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1092/2210] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:16.969024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:16.980462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848f1a9-ae4a-49cd-802d-caa7e5dd9b44 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:16.986688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee851d1-2bc7-462b-a4f1-002d98d0850b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:16.987630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee851d1-2bc7-462b-a4f1-002d98d0850b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:16.988124 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee851d1-2bc7-462b-a4f1-002d98d0850b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:16.989036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee851d1-2bc7-462b-a4f1-002d98d0850b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:17.004851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee851d1-2bc7-462b-a4f1-002d98d0850b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:17.005953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee851d1-2bc7-462b-a4f1-002d98d0850b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:17.012563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:17.021828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1119/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:21.468229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:21.560588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2e0a6-3132-4083-a672-18a9ae1a3544 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:21.565501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:21.566196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.566477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.566753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] PATH_INFO: `/v3/users/ce01aa1b29dd41fcab68765154234e6c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.567961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce01aa1b29dd41fcab68765154234e6c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:21.570125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:21.635369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce01aa1b29dd41fcab68765154234e6c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:21.648933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce01aa1b29dd41fcab68765154234e6c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:21.673071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce01aa1b29dd41fcab68765154234e6c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:21.674535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce01aa1b29dd41fcab68765154234e6c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:21.688249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce01aa1b29dd41fcab68765154234e6c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:21.702627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1093/2212] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:09:21 2022] DELETE /identity/v3/users/ce01aa1b29dd41fcab68765154234e6c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:21.705115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.706037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.708625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.734140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:21.737270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:21.843217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1120/2213] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:21.854464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:21.857522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:21.967406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf56eae-20bf-43da-8d67-54bdac0c4436 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:21.980547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:21.981855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.982371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.983136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.013409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.016439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:22.041048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1094/2214] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:22.449792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:22.454139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:22.660077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8d0a6-d36b-41ce-9e65-9e1ccc97c60a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.668536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.671018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.673872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.673872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.679522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:22.705137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.712047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:22.753248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1121/2215] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:22.850713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:22.853159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:22.932206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17896-502b-48b8-878b-fcfd1e6f3024 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.958373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.959478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.959978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.960457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] PATH_INFO: `/v3/projects/413f90a33acf4f768282cf2288795728` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.969170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=413f90a33acf4f768282cf2288795728)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.977595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:23.018702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.119037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413f90a33acf4f768282cf2288795728', 'request_id': 'req-92d36522-5c7b-408b-91fc-344b84e6b23b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.127027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413f90a33acf4f768282cf2288795728', 'request_id': 'req-92d36522-5c7b-408b-91fc-344b84e6b23b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.136292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413f90a33acf4f768282cf2288795728', 'request_id': 'req-92d36522-5c7b-408b-91fc-344b84e6b23b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.171091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] The token cache is being invalidate because project 413f90a33acf4f768282cf2288795728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:09:23.171380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.172073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.174330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1095/2216] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:09:22 2022] DELETE /identity/v3/projects/413f90a33acf4f768282cf2288795728 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:23.178379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.190202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fed084-3e62-4655-a5b5-f8694e24e699 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.334703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.337061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.337549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.338073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] PATH_INFO: `/v3/users/e0eb300e662d4908bb23689e407667c6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.340574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0eb300e662d4908bb23689e407667c6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:23.344631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:23.440491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.469070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0eb300e662d4908bb23689e407667c6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.484580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0eb300e662d4908bb23689e407667c6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.494303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0eb300e662d4908bb23689e407667c6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.495132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0eb300e662d4908bb23689e407667c6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.519805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0eb300e662d4908bb23689e407667c6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:23.528139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1122/2217] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:09:23 2022] DELETE /identity/v3/users/e0eb300e662d4908bb23689e407667c6 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:25.064894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.069504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:25.110639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36522-5c7b-408b-91fc-344b84e6b23b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.284663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.286158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.286453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.288172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] PATH_INFO: `/v3/projects/513f1afb1b05478e8b27af897703e429` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.292457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=513f1afb1b05478e8b27af897703e429)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.294860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.447104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513f1afb1b05478e8b27af897703e429', 'request_id': 'req-dadc04ce-a226-4de4-8674-9219920c40c9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.461602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513f1afb1b05478e8b27af897703e429', 'request_id': 'req-dadc04ce-a226-4de4-8674-9219920c40c9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.478389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513f1afb1b05478e8b27af897703e429', 'request_id': 'req-dadc04ce-a226-4de4-8674-9219920c40c9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.498306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] The token cache is being invalidate because project 513f1afb1b05478e8b27af897703e429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:09:25.498647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.499298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.500868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1096/2218] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:09:25 2022] DELETE /identity/v3/projects/513f1afb1b05478e8b27af897703e429 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:25.505087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.506871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:25.520838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee831894-21e0-4912-bc9e-edcd257e6c89 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.613477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.614040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.614597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.614918 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] PATH_INFO: `/v3/users/101a8fb5057d466fb0097c0f2539664b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.617462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101a8fb5057d466fb0097c0f2539664b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.620898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.695633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101a8fb5057d466fb0097c0f2539664b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.717366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101a8fb5057d466fb0097c0f2539664b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.733042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101a8fb5057d466fb0097c0f2539664b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.734233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101a8fb5057d466fb0097c0f2539664b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.751175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9427e8bf-2214-4727-a1e7-37dae48d5723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101a8fb5057d466fb0097c0f2539664b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:25.764392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1123/2219] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:09:25 2022] DELETE /identity/v3/users/101a8fb5057d466fb0097c0f2539664b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:26.375535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.390139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:26.420237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc04ce-a226-4de4-8674-9219920c40c9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.485024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:26.532994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.534115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.534572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.535034 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] PATH_INFO: `/v3/projects/6987bab936d444aa815a8bf7571355c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.537276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6987bab936d444aa815a8bf7571355c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.541019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.622689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6987bab936d444aa815a8bf7571355c0', 'request_id': 'req-edab0a81-d64b-4119-a597-d86f496a12e5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:26.639462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6987bab936d444aa815a8bf7571355c0', 'request_id': 'req-edab0a81-d64b-4119-a597-d86f496a12e5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:26.651328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6987bab936d444aa815a8bf7571355c0', 'request_id': 'req-edab0a81-d64b-4119-a597-d86f496a12e5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:26.675199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] The token cache is being invalidate because project 6987bab936d444aa815a8bf7571355c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:09:26.683422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:26.683422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-edab0a81-d64b-4119-a597-d86f496a12e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:09:26.683422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1097/2220] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:09:26 2022] DELETE /identity/v3/projects/6987bab936d444aa815a8bf7571355c0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:26.688112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986a569-8e50-4401-bee8-e24c088afdc2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.689102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986a569-8e50-4401-bee8-e24c088afdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.689379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986a569-8e50-4401-bee8-e24c088afdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.715598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4986a569-8e50-4401-bee8-e24c088afdc2 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:26.716555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4986a569-8e50-4401-bee8-e24c088afdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:26.752398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:26.804227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1124/2221] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:26.814950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9daf-3cde-42c6-a216-317f78bfc823 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.815504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9daf-3cde-42c6-a216-317f78bfc823 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.816000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9daf-3cde-42c6-a216-317f78bfc823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.844497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f1df9daf-3cde-42c6-a216-317f78bfc823 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:26.845850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f1df9daf-3cde-42c6-a216-317f78bfc823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:26.854774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1098/2222] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:26.860207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986a569-8e50-4401-bee8-e24c088afdc2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.879248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986a569-8e50-4401-bee8-e24c088afdc2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.887738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.888770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.889256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.889770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.890980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:26.891506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.894641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.914863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1125/2223] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:26.920417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df9daf-3cde-42c6-a216-317f78bfc823 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.932003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df9daf-3cde-42c6-a216-317f78bfc823 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.938488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.939510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.939847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.940254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.941317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.945388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.984026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1099/2224] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:09:26 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:26.989611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.005727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec141-e1eb-42a1-989e-0dff99cc25eb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.102260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.103277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.103652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.104022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.104779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.107424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.141796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1126/2225] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:09:26 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:09:27.146033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.164378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d633a-8457-4c88-89af-17ef9098e154 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.173453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.174537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.175031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.175522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.176781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.180416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.217959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1100/2226] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:27.218227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.228508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89eaa4e-e6f2-4116-9f43-889de783a0ef admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.235303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156645c0-f694-44ca-9ea4-1125efa5ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.235947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156645c0-f694-44ca-9ea4-1125efa5ecd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.236227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156645c0-f694-44ca-9ea4-1125efa5ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.236490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156645c0-f694-44ca-9ea4-1125efa5ecd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.237154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156645c0-f694-44ca-9ea4-1125efa5ecd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.239036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156645c0-f694-44ca-9ea4-1125efa5ecd9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.253884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1127/2227] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:09:27.256434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.268219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351b1c-ba1f-4e4f-bb43-9b878e6a7c9b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.278684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.278684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.278684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.278684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] PATH_INFO: `/v3/projects/796478c0ae924c9ab77e868da74ae56b/users/a228f6738de74faabc99e9109347f7bd/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.312022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796478c0ae924c9ab77e868da74ae56b, user_id=a228f6738de74faabc99e9109347f7bd, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.317927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.353701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1101/2228] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:09:27 2022] PUT /identity/v3/projects/796478c0ae924c9ab77e868da74ae56b/users/a228f6738de74faabc99e9109347f7bd/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:27.443067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02c08-478c-4386-9779-f00529e796be None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.443629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02c08-478c-4386-9779-f00529e796be None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.444129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02c08-478c-4386-9779-f00529e796be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.511980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f6f02c08-478c-4386-9779-f00529e796be None None] MFA Rules not processed for user `a228f6738de74faabc99e9109347f7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:27.515431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f6f02c08-478c-4386-9779-f00529e796be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:27.676492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1128/2229] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:09:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:27.691603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.694700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:27.820781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54346d03-1263-4037-b3c4-401d6fcb5f99 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.826213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.827020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.827485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.827899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.839772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.841976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.856202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1102/2230] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:37.668163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:37.850783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:05.900232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15f8b-bdd4-406b-9804-0e6e3f7e5877 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:05.901228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15f8b-bdd4-406b-9804-0e6e3f7e5877 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:05.901228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15f8b-bdd4-406b-9804-0e6e3f7e5877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.930193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c1b15f8b-bdd4-406b-9804-0e6e3f7e5877 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:05.933077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c1b15f8b-bdd4-406b-9804-0e6e3f7e5877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.065049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1129/2231] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:10:06.147640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:06.151068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.248490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04f8ae-5ab4-4699-838a-9d621d5e39a8 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:06.259292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451bca0-df26-4921-ad8c-b40208fdfe4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:06.260431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451bca0-df26-4921-ad8c-b40208fdfe4a service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.260999 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451bca0-df26-4921-ad8c-b40208fdfe4a service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.261539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451bca0-df26-4921-ad8c-b40208fdfe4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.282096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451bca0-df26-4921-ad8c-b40208fdfe4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:06.283902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451bca0-df26-4921-ad8c-b40208fdfe4a service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:06.299895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1103/2232] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:10:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:10:15.548162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffe2a0-f54d-4368-8687-d36a86f63a8d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:15.549217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffe2a0-f54d-4368-8687-d36a86f63a8d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:15.549483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffe2a0-f54d-4368-8687-d36a86f63a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:15.588113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-78ffe2a0-f54d-4368-8687-d36a86f63a8d None None] MFA Rules not processed for user `ca4779abc6024a0f84f041ca7c058e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:15.590291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-78ffe2a0-f54d-4368-8687-d36a86f63a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:15.701759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1130/2233] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:10:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:10:16.292698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=32 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:25.687716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=50 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:42.356391 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:42.356957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:42.357117 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:42.411244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:42.412887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:42.422578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1104/2234] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 13:10:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:42.439676 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffe2a0-f54d-4368-8687-d36a86f63a8d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:42.458360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffe2a0-f54d-4368-8687-d36a86f63a8d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:42.467820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:42.468926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:42.469396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:42.469908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:42.486577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:42.489517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:42.509638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1131/2235] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:10:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:10:52.393269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:52.394954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:52.482712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc269122-4823-4635-bb41-6bd0f95237cb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.489462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.490313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.490683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.491138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] PATH_INFO: `/v3/users/86032a9fd3b8429b99a7d23ddaf03116` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.492635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] RBAC: Authorizing `identity:delete_user(user_id=86032a9fd3b8429b99a7d23ddaf03116)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:52.495299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:52.500674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:52.563724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86032a9fd3b8429b99a7d23ddaf03116'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:52.574381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86032a9fd3b8429b99a7d23ddaf03116'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:52.590241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86032a9fd3b8429b99a7d23ddaf03116'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:52.591389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86032a9fd3b8429b99a7d23ddaf03116'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:52.603832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d8fbd78-9410-4a92-b7ec-2d58b40f642d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86032a9fd3b8429b99a7d23ddaf03116'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:52.614708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1105/2236] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:10:52 2022] DELETE /identity/v3/users/86032a9fd3b8429b99a7d23ddaf03116 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:10:52.829097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:52.844288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7caf3c-6591-4a2e-b480-b15a74c78f8e service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.936103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.937222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.937956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.938327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] PATH_INFO: `/v3/projects/4da3acfcc34b44a5909e036705c040c8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.939816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da3acfcc34b44a5909e036705c040c8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:52.942385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.013860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da3acfcc34b44a5909e036705c040c8', 'request_id': 'req-fb219dc1-6c73-431f-9e9a-1738ea11fb64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:53.020662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da3acfcc34b44a5909e036705c040c8', 'request_id': 'req-fb219dc1-6c73-431f-9e9a-1738ea11fb64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:53.034175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da3acfcc34b44a5909e036705c040c8', 'request_id': 'req-fb219dc1-6c73-431f-9e9a-1738ea11fb64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:53.059621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] The token cache is being invalidate because project 4da3acfcc34b44a5909e036705c040c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:10:53.060136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:53.063032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fb219dc1-6c73-431f-9e9a-1738ea11fb64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:10:53.066027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1132/2237] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:10:52 2022] DELETE /identity/v3/projects/4da3acfcc34b44a5909e036705c040c8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:10:53.075379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46b8e3-e047-4d89-a7f1-c2107ad638b9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.075690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46b8e3-e047-4d89-a7f1-c2107ad638b9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.076161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46b8e3-e047-4d89-a7f1-c2107ad638b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.112081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-de46b8e3-e047-4d89-a7f1-c2107ad638b9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:53.114307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-de46b8e3-e047-4d89-a7f1-c2107ad638b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:53.229352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1106/2238] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:53.242827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9578a3-e8a9-487e-b8f5-20a4ab4b1d0f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.243402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9578a3-e8a9-487e-b8f5-20a4ab4b1d0f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.243876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9578a3-e8a9-487e-b8f5-20a4ab4b1d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.273093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-da9578a3-e8a9-487e-b8f5-20a4ab4b1d0f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:53.274533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-da9578a3-e8a9-487e-b8f5-20a4ab4b1d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:53.288804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1133/2239] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:53.294614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46b8e3-e047-4d89-a7f1-c2107ad638b9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.306688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46b8e3-e047-4d89-a7f1-c2107ad638b9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.311595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.312429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.312716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.312959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.313625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:53.313976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.315872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.326818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:53.335845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1107/2240] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:53.337909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9578a3-e8a9-487e-b8f5-20a4ab4b1d0f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.350156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9578a3-e8a9-487e-b8f5-20a4ab4b1d0f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.356358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.357200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.357489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.357781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.358782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.361298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.407392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1134/2241] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:10:53 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:53.408946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.433029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcace75c-5cd4-4bdf-a49d-3cb1b2042b4e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.514244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.514881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.515169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.515449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.516125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.518181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.544067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1108/2242] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:53 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:10:53.549792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.564461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e01fc3-45b2-4b72-8e8d-e4dedb08fe6e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.574902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.575692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.576064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.576510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.577479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.579588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.594285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1135/2243] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:10:53.600198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.615205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca961f-668b-44c3-b833-bd2683345619 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.622332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69584639-fd18-4b16-a6eb-8bf183d8885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.623133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69584639-fd18-4b16-a6eb-8bf183d8885d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.623431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69584639-fd18-4b16-a6eb-8bf183d8885d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.623792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69584639-fd18-4b16-a6eb-8bf183d8885d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.624482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69584639-fd18-4b16-a6eb-8bf183d8885d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.626180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69584639-fd18-4b16-a6eb-8bf183d8885d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.640780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1109/2244] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:10:53.645585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.657335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f41fb7-120c-4316-b02f-c9a77faea7a5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.663538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.664261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.664562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.665023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] PATH_INFO: `/v3/projects/b2781e0fdbdb4a8d850015c284b684cb/users/cfb20f6454ca4244b3eed22543c0e63f/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.685025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2781e0fdbdb4a8d850015c284b684cb, user_id=cfb20f6454ca4244b3eed22543c0e63f, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.688049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.703871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1136/2245] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:10:53 2022] PUT /identity/v3/projects/b2781e0fdbdb4a8d850015c284b684cb/users/cfb20f6454ca4244b3eed22543c0e63f/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:10:53.716643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79caa00-b4dd-4d61-90b0-23362d7bb2ed None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.716939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79caa00-b4dd-4d61-90b0-23362d7bb2ed None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.717398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79caa00-b4dd-4d61-90b0-23362d7bb2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.762356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d79caa00-b4dd-4d61-90b0-23362d7bb2ed None None] MFA Rules not processed for user `cfb20f6454ca4244b3eed22543c0e63f`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:53.763998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d79caa00-b4dd-4d61-90b0-23362d7bb2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:53.854888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1110/2246] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:53.869554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.871544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:53.910934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:53.978807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f2ed-25a3-4408-a3d4-168e47ca1437 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.983450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.984064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.984317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.984585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.997317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.998888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:54.010118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1137/2247] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:10:57.985048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:57.985610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:57.985610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.005906 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:58.007632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:58.101181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1111/2248] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:10:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:58.135334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:58.137956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:58.261956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51202c-3bd6-4abd-81e1-d2558cd23863 service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:58.267475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7ad47-7199-4451-9fc3-e2fdde8023dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:58.268114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ad47-7199-4451-9fc3-e2fdde8023dc service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:58.268395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ad47-7199-4451-9fc3-e2fdde8023dc service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:58.268666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ad47-7199-4451-9fc3-e2fdde8023dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.280804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7ad47-7199-4451-9fc3-e2fdde8023dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:58.282712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7ad47-7199-4451-9fc3-e2fdde8023dc service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:58.294115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1138/2249] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:10:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:11:04.059961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.062609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:04.105649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:04.234245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee655c8-80d5-4d90-8c60-c3c0bf8d2fdd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.244599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.246143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.246644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.247135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] PATH_INFO: `/v3/users/a228f6738de74faabc99e9109347f7bd` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.249413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a228f6738de74faabc99e9109347f7bd)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:04.252899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:04.470860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a228f6738de74faabc99e9109347f7bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:04.487993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a228f6738de74faabc99e9109347f7bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:04.542840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a228f6738de74faabc99e9109347f7bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:04.544104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a228f6738de74faabc99e9109347f7bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:04.561840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a228f6738de74faabc99e9109347f7bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:04.574476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1112/2250] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:11:04 2022] DELETE /identity/v3/users/a228f6738de74faabc99e9109347f7bd => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:11:04.580033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.580834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.580997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.676835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:04.678737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:04.918479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1139/2251] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:11:04.927313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.931019 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:05.073213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2022-4b22-42fa-baae-9f4c75380218 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:05.079810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b0a92-a2a1-4430-b677-46d17a2bd1e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:05.080509 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0a92-a2a1-4430-b677-46d17a2bd1e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:05.080883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0a92-a2a1-4430-b677-46d17a2bd1e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:05.081215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0a92-a2a1-4430-b677-46d17a2bd1e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:05.093419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b0a92-a2a1-4430-b677-46d17a2bd1e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:05.095010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b0a92-a2a1-4430-b677-46d17a2bd1e9 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:05.108694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1113/2252] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:11:05.647479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:05.651402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:05.672048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ce897-b26b-487a-a7f2-0e928deec1e6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:05.682712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:05.685087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:05.685950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:05.686565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] PATH_INFO: `/v3/projects/796478c0ae924c9ab77e868da74ae56b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:05.688866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=796478c0ae924c9ab77e868da74ae56b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:05.692543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:05.838320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:05.884093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796478c0ae924c9ab77e868da74ae56b', 'request_id': 'req-a8148914-7e15-423b-a2e1-79bfec7fa222'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:05.901248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796478c0ae924c9ab77e868da74ae56b', 'request_id': 'req-a8148914-7e15-423b-a2e1-79bfec7fa222'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:05.919639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796478c0ae924c9ab77e868da74ae56b', 'request_id': 'req-a8148914-7e15-423b-a2e1-79bfec7fa222'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:05.938359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] The token cache is being invalidate because project 796478c0ae924c9ab77e868da74ae56b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:11:05.938673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:05.939403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a8148914-7e15-423b-a2e1-79bfec7fa222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:05.942168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1140/2253] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:11:05 2022] DELETE /identity/v3/projects/796478c0ae924c9ab77e868da74ae56b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:11:05.950630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b79b-72c4-4c18-8a16-031d7273ab2a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:05.951559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b79b-72c4-4c18-8a16-031d7273ab2a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:05.952326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b79b-72c4-4c18-8a16-031d7273ab2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.039281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cde4b79b-72c4-4c18-8a16-031d7273ab2a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:06.042733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cde4b79b-72c4-4c18-8a16-031d7273ab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:06.126270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:06.210535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99215a87-e886-47d2-8407-17d647e18add None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.211000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99215a87-e886-47d2-8407-17d647e18add None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.211423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99215a87-e886-47d2-8407-17d647e18add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.219324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1114/2254] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:11:06.319341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-99215a87-e886-47d2-8407-17d647e18add None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:06.320709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-99215a87-e886-47d2-8407-17d647e18add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:06.339980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1141/2255] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:11:06.344590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4b79b-72c4-4c18-8a16-031d7273ab2a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.355348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4b79b-72c4-4c18-8a16-031d7273ab2a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.360441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.361065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.362367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.362660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.363588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:06.365061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.368496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.402934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1115/2256] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:06.407032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99215a87-e886-47d2-8407-17d647e18add None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.423709 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99215a87-e886-47d2-8407-17d647e18add None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.455232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.456278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.456756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.457202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.459113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.461159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.523337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1142/2257] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:11:06 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:11:06.525298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.539658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c13081-d216-4868-92a9-289c3f48bc6f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.672565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.673143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.673499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.673882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.674634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.676841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.725942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1116/2258] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:11:06 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:11:06.727266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.742071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28721b-0f2b-4ee2-94be-882a668931b5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.751549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.752227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.752540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.752818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.753483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.755353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.769713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1143/2259] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:11:06.774879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.792298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198cdf9-f07d-4c49-8ab7-d96b1b8ffe57 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.798045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1651c16-95ed-40f2-aea5-ba98a8671293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.798976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1651c16-95ed-40f2-aea5-ba98a8671293 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.799097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1651c16-95ed-40f2-aea5-ba98a8671293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.802229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1651c16-95ed-40f2-aea5-ba98a8671293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.802229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1651c16-95ed-40f2-aea5-ba98a8671293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.802229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1651c16-95ed-40f2-aea5-ba98a8671293 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.816172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1117/2260] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:11:06.820428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.835562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14fbe-b705-4ab5-abd6-1566acb78180 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.841964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.844079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.844890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.845500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] PATH_INFO: `/v3/projects/39d21dc0c5c54a1eb456d0a6d32b36de/users/9516dfea8ad44fc2895653e0500ac43f/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.886461 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d21dc0c5c54a1eb456d0a6d32b36de, user_id=9516dfea8ad44fc2895653e0500ac43f, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.889341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.933710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1144/2261] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:11:06 2022] PUT /identity/v3/projects/39d21dc0c5c54a1eb456d0a6d32b36de/users/9516dfea8ad44fc2895653e0500ac43f/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:11:06.938311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73354455-7206-4d95-b239-d61ddc437caa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.938911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73354455-7206-4d95-b239-d61ddc437caa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.939358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73354455-7206-4d95-b239-d61ddc437caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:07.003272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-73354455-7206-4d95-b239-d61ddc437caa None None] MFA Rules not processed for user `9516dfea8ad44fc2895653e0500ac43f`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:07.004702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-73354455-7206-4d95-b239-d61ddc437caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:07.118124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1118/2262] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:11:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:11:07.136275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:07.138751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:07.237990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b907d-838d-4763-a2b9-85de34ac2e0a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:07.244111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93a5ea-eebe-4aa2-8d69-4039c6f890d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:07.246086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93a5ea-eebe-4aa2-8d69-4039c6f890d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:07.246911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93a5ea-eebe-4aa2-8d69-4039c6f890d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:07.247620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93a5ea-eebe-4aa2-8d69-4039c6f890d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:07.260214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93a5ea-eebe-4aa2-8d69-4039c6f890d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:07.262511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93a5ea-eebe-4aa2-8d69-4039c6f890d9 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:07.285101 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1145/2263] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:07.367249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73354455-7206-4d95-b239-d61ddc437caa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:07.484717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:07.575075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73354455-7206-4d95-b239-d61ddc437caa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:07.597721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:07.597721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:07.597721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:07.597721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] PATH_INFO: `/v3/users/9516dfea8ad44fc2895653e0500ac43f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:07.598706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9516dfea8ad44fc2895653e0500ac43f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:07.600788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:07.718450 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9516dfea8ad44fc2895653e0500ac43f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:07.731435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9516dfea8ad44fc2895653e0500ac43f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:07.791403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9516dfea8ad44fc2895653e0500ac43f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:07.793926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9516dfea8ad44fc2895653e0500ac43f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:07.808110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9516dfea8ad44fc2895653e0500ac43f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:07.818424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1119/2264] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:11:07 2022] DELETE /identity/v3/users/9516dfea8ad44fc2895653e0500ac43f => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:11:07.825897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:07.826716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:07.826716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:07.879199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:07.881423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:08.069804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1146/2265] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:11:08.083602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:08.086207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:08.228445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc150a75-e208-4636-992c-d192835ffda4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:08.233965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:08.235010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:08.235415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:08.235877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:08.253572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:08.259604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:08.279912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1120/2266] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:08.823348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:08.826187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:08.849261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:08.859532 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a1f0-b4cd-41ea-94c3-784ae65f10b3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:08.868812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:08.870184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:08.870761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:08.871554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] PATH_INFO: `/v3/projects/39d21dc0c5c54a1eb456d0a6d32b36de` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:08.873393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d21dc0c5c54a1eb456d0a6d32b36de)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:08.876002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:09.000842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d21dc0c5c54a1eb456d0a6d32b36de', 'request_id': 'req-247ae829-ea39-434a-9adf-6109e7bebe03'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:09.029618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d21dc0c5c54a1eb456d0a6d32b36de', 'request_id': 'req-247ae829-ea39-434a-9adf-6109e7bebe03'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:09.044850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d21dc0c5c54a1eb456d0a6d32b36de', 'request_id': 'req-247ae829-ea39-434a-9adf-6109e7bebe03'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:09.067530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] The token cache is being invalidate because project 39d21dc0c5c54a1eb456d0a6d32b36de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:11:09.068271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:09.069023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:11:09.073743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1147/2267] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:11:08 2022] DELETE /identity/v3/projects/39d21dc0c5c54a1eb456d0a6d32b36de => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:11:14.423233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:14.424063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:14.437550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823562b4-446f-4067-9cf1-55061a467aba service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:14.508026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43eb352-9159-442f-b901-e6bdf0f33c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:14.508706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43eb352-9159-442f-b901-e6bdf0f33c65 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:14.508978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43eb352-9159-442f-b901-e6bdf0f33c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:14.509314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43eb352-9159-442f-b901-e6bdf0f33c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:14.511808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a43eb352-9159-442f-b901-e6bdf0f33c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:14.518557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:14.626300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43eb352-9159-442f-b901-e6bdf0f33c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:14.628105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43eb352-9159-442f-b901-e6bdf0f33c65 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:14.651915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1121/2268] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:15.875947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:15.878380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:15.895793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ae829-ea39-434a-9adf-6109e7bebe03 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:15.900168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:15.900842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:15.901170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:15.901451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:15.904024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:15.921442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:15.923188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:15.937107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1148/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:24.635953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=30 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:25.121043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:25.121310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:25.121581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:25.156334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] MFA Rules not processed for user `b22dd3b0461747688993851ccf4e66d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:25.161016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:25.284868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1122/2270] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:11:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:11:25.653348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:25.662164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:25.677662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78464e3e-55f0-4e6d-b7a0-20d093cdaac3 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:25.684137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814af941-79ba-4bf1-9c61-a1b03d43b105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:25.685003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af941-79ba-4bf1-9c61-a1b03d43b105 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:25.685512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af941-79ba-4bf1-9c61-a1b03d43b105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:25.686039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af941-79ba-4bf1-9c61-a1b03d43b105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:25.689436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-814af941-79ba-4bf1-9c61-a1b03d43b105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:25.714509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af941-79ba-4bf1-9c61-a1b03d43b105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:25.723442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af941-79ba-4bf1-9c61-a1b03d43b105 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:25.753193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1149/2271] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:26.211699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:26.213803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:26.314839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77b5e-1d82-4f44-ad2b-3016d1bc9da5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:26.320926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38606c94-beea-483d-ab82-755bf2cedbab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:26.321568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38606c94-beea-483d-ab82-755bf2cedbab service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:26.322134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38606c94-beea-483d-ab82-755bf2cedbab service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:26.322428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38606c94-beea-483d-ab82-755bf2cedbab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:26.324487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-38606c94-beea-483d-ab82-755bf2cedbab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:26.337596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38606c94-beea-483d-ab82-755bf2cedbab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:26.339281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38606c94-beea-483d-ab82-755bf2cedbab service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:26.351865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1123/2272] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:26.981119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830567a3-118f-46a8-837d-b0e28e1f853e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:26.981981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830567a3-118f-46a8-837d-b0e28e1f853e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:26.982384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830567a3-118f-46a8-837d-b0e28e1f853e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:26.985260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1150/2273] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:11:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:26.993816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13773a2a-3178-431d-a913-7cdebc3bcc17 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:26.995855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13773a2a-3178-431d-a913-7cdebc3bcc17 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:26.995855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13773a2a-3178-431d-a913-7cdebc3bcc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.034322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-13773a2a-3178-431d-a913-7cdebc3bcc17 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:27.036398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-13773a2a-3178-431d-a913-7cdebc3bcc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:27.051204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1124/2274] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:11:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:11:27.070411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830567a3-118f-46a8-837d-b0e28e1f853e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:27.074038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-830567a3-118f-46a8-837d-b0e28e1f853e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:27.183257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830567a3-118f-46a8-837d-b0e28e1f853e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:27.192251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f6b36-264d-4152-911c-21959ea7f865 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:27.192938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f6b36-264d-4152-911c-21959ea7f865 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:27.193259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f6b36-264d-4152-911c-21959ea7f865 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:27.193556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f6b36-264d-4152-911c-21959ea7f865 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.217956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f6b36-264d-4152-911c-21959ea7f865 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:27.221229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f6b36-264d-4152-911c-21959ea7f865 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:27.247882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1151/2275] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:27.365400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281660c-a935-4186-b97c-ce15b74466b8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:27.366160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281660c-a935-4186-b97c-ce15b74466b8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:27.366160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281660c-a935-4186-b97c-ce15b74466b8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.368341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1125/2276] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:11:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:27.375433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8883-3677-494c-b65e-7001136c6c85 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:27.382420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8883-3677-494c-b65e-7001136c6c85 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:27.382772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8883-3677-494c-b65e-7001136c6c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.410285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b98a8883-3677-494c-b65e-7001136c6c85 None None] MFA Rules not processed for user `4302574365514eb1b25bd8e62736a4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:27.411546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b98a8883-3677-494c-b65e-7001136c6c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:27.418964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1152/2277] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:11:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:11:27.436169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281660c-a935-4186-b97c-ce15b74466b8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:27.458691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281660c-a935-4186-b97c-ce15b74466b8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:27.463073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886051-adea-4c85-93b9-3bd014fe5c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:27.463790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886051-adea-4c85-93b9-3bd014fe5c3f service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:27.464302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886051-adea-4c85-93b9-3bd014fe5c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:27.464617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886051-adea-4c85-93b9-3bd014fe5c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.474700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce886051-adea-4c85-93b9-3bd014fe5c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:27.476318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce886051-adea-4c85-93b9-3bd014fe5c3f service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:27.487925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1126/2278] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:27.620443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741332-dc27-40f1-bc6c-5830b254d346 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:27.620934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741332-dc27-40f1-bc6c-5830b254d346 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:27.621394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741332-dc27-40f1-bc6c-5830b254d346 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.623980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1153/2279] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:11:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:27.634630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:27.634630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:27.634630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.677779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:27.679254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:27.697467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1127/2280] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:11:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:11:27.710473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf741332-dc27-40f1-bc6c-5830b254d346 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:27.727834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf741332-dc27-40f1-bc6c-5830b254d346 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:27.736381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:27.737332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:27.737801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:27.738233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:27.756341 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:27.758741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:27.775564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1154/2281] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:35.956141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:35.958695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:35.983100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fd9b5-a0b5-4b36-afaf-e45e9d70007d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:35.989212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:35.990091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:35.990569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:35.990916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:35.992987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:36.105494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:36.108577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:36.124940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1128/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:37.766720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:40.362457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:40.370945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:40.380083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5de3e-3cc7-48fe-9916-eb25cbeaba1b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:40.389251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c815-75ba-460e-b8af-e6a5531b2b3a service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:40.391042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:40.392123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:40.392491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:40.393216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:40.394752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de8955-e9a1-4d22-acf0-faefead1cf0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:40.396405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de8955-e9a1-4d22-acf0-faefead1cf0c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:40.396759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de8955-e9a1-4d22-acf0-faefead1cf0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:40.397277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de8955-e9a1-4d22-acf0-faefead1cf0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:40.401884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-38de8955-e9a1-4d22-acf0-faefead1cf0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:40.407122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:40.415132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:40.423441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de8955-e9a1-4d22-acf0-faefead1cf0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:40.425322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:40.425567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de8955-e9a1-4d22-acf0-faefead1cf0c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:40.428311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:40.438152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1129/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:11:40.449892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1155/2284] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:11:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:11:41.321544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:41.322769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:41.322769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:41.390883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:41.393444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:41.547326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1130/2285] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:11:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:11:41.575493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:41.578349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:41.702044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020c5e-1099-4d4d-8e6a-f745622fb483 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:41.708232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d5d15-a7b7-4334-8439-c0b9575e97c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:41.708994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d5d15-a7b7-4334-8439-c0b9575e97c2 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:41.709331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d5d15-a7b7-4334-8439-c0b9575e97c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:41.709603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d5d15-a7b7-4334-8439-c0b9575e97c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:41.720691 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d5d15-a7b7-4334-8439-c0b9575e97c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:41.722760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d5d15-a7b7-4334-8439-c0b9575e97c2 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:41.737243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1156/2286] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:11:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:11:51.533439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=28 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:51.731743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=38 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:05.868078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:05.870657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:05.996726 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff27bb-adbb-4c21-9c88-334495692fcd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:06.004818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:06.006091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:06.006594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:06.007090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] PATH_INFO: `/v3/users/cfb20f6454ca4244b3eed22543c0e63f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:06.009183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb20f6454ca4244b3eed22543c0e63f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:06.013050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:06.100434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb20f6454ca4244b3eed22543c0e63f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:06.113633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb20f6454ca4244b3eed22543c0e63f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:06.139525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb20f6454ca4244b3eed22543c0e63f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:06.140294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb20f6454ca4244b3eed22543c0e63f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:06.149056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb20f6454ca4244b3eed22543c0e63f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:06.158930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1131/2287] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:12:05 2022] DELETE /identity/v3/users/cfb20f6454ca4244b3eed22543c0e63f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:12:06.165370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:06.166355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:06.166355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:06.201394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:06.202770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:06.317114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1157/2288] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:12:06.331942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:06.334287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:06.496859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9de49-71cb-40ff-b59a-51da877142e0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:06.502657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:06.503546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:06.503951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:06.504327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:06.518643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:06.521268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:06.545304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1132/2289] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:12:07.042377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:07.045440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:07.061733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223582d0-9fb1-4ea7-b188-218b23405446 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:07.067020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:07.067781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:07.068139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:07.068552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] PATH_INFO: `/v3/projects/b2781e0fdbdb4a8d850015c284b684cb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:07.069567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2781e0fdbdb4a8d850015c284b684cb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:07.071981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:07.182544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2781e0fdbdb4a8d850015c284b684cb', 'request_id': 'req-f6cda3c8-c5be-490d-985d-c90d5a997bfe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:07.191839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2781e0fdbdb4a8d850015c284b684cb', 'request_id': 'req-f6cda3c8-c5be-490d-985d-c90d5a997bfe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:07.203478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2781e0fdbdb4a8d850015c284b684cb', 'request_id': 'req-f6cda3c8-c5be-490d-985d-c90d5a997bfe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:07.222836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] The token cache is being invalidate because project b2781e0fdbdb4a8d850015c284b684cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:12:07.223168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:07.223963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6cda3c8-c5be-490d-985d-c90d5a997bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:07.225983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1158/2290] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:12:07 2022] DELETE /identity/v3/projects/b2781e0fdbdb4a8d850015c284b684cb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:12:13.308149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:13.310195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:13.362536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:13.438543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6078-bce8-4317-87d7-3b95365ab781 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:13.557116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:13.557663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:13.558021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:13.558805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:13.560548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:13.582946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:13.584950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:13.598021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1133/2291] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:12:17.217740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:22.280562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:22.281783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:22.281783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:22.315912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:22.318058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:22.444546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1159/2292] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 13:12:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:12:22.476924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:22.479513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:22.630312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63113a7-9d66-4c32-bd14-5901dd4bcb86 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:22.639730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51af4f7-4391-42af-93a5-94feae43138c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:22.640447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af4f7-4391-42af-93a5-94feae43138c service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:22.640730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af4f7-4391-42af-93a5-94feae43138c service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:22.641004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af4f7-4391-42af-93a5-94feae43138c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:22.653110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51af4f7-4391-42af-93a5-94feae43138c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:22.655658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51af4f7-4391-42af-93a5-94feae43138c service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:22.677223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1134/2293] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:12:32.430773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=25 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:32.663238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:38.423745 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:38.426136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:38.445421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def88303-56bf-4a8c-8c28-c1abff2e62e2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:38.454014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa815e5-0556-4ab4-baad-988d92a941e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:38.454656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa815e5-0556-4ab4-baad-988d92a941e9 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:38.454920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa815e5-0556-4ab4-baad-988d92a941e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:38.455186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa815e5-0556-4ab4-baad-988d92a941e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:38.457030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8fa815e5-0556-4ab4-baad-988d92a941e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:38.477020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa815e5-0556-4ab4-baad-988d92a941e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:38.479996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa815e5-0556-4ab4-baad-988d92a941e9 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:38.498694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1160/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:12:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:12:48.490110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:10.131940 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd09b4-7d6a-44a9-b67f-01c7d59bd565 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:10.132895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd09b4-7d6a-44a9-b67f-01c7d59bd565 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:10.133057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd09b4-7d6a-44a9-b67f-01c7d59bd565 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:10.138202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1135/2295] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:13:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:13:10.168801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:10.170007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:10.170496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:10.215757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] MFA Rules not processed for user `1d0c0be08934466c858ee214b06538b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:10.218697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:10.255028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd09b4-7d6a-44a9-b67f-01c7d59bd565 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:10.258238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2bbd09b4-7d6a-44a9-b67f-01c7d59bd565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:10.358414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1161/2296] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:13:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:13:10.378904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:10.381264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:10.389962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd09b4-7d6a-44a9-b67f-01c7d59bd565 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:10.397337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:10.398305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:10.398765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:10.398929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158efce-999b-4827-b9bb-4347b63a8bc5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:10.399137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] PATH_INFO: `/v3/users/b22dd3b0461747688993851ccf4e66d0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:10.400992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22dd3b0461747688993851ccf4e66d0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:10.404767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:10.405136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:10.405860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:10.406149 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:10.406423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:10.419489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:10.421521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:10.436200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1162/2297] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:13:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:13:10.496178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22dd3b0461747688993851ccf4e66d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:10.508388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22dd3b0461747688993851ccf4e66d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:10.531465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22dd3b0461747688993851ccf4e66d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:10.532258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22dd3b0461747688993851ccf4e66d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:10.541553 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22dd3b0461747688993851ccf4e66d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:10.548508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1136/2298] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:13:10 2022] DELETE /identity/v3/users/b22dd3b0461747688993851ccf4e66d0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:13:10.781229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:10.801040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa19dd0-f480-439a-971c-fd336188f0f9 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:10.894775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:10.895450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:10.895784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:10.896092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] PATH_INFO: `/v3/projects/73afbf716079452ba1deb906dbdce949` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:10.897316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73afbf716079452ba1deb906dbdce949)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:10.899285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:10.973206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73afbf716079452ba1deb906dbdce949', 'request_id': 'req-d15ae614-3212-4841-898b-85a98ba12041'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:10.980256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73afbf716079452ba1deb906dbdce949', 'request_id': 'req-d15ae614-3212-4841-898b-85a98ba12041'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:10.991736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73afbf716079452ba1deb906dbdce949', 'request_id': 'req-d15ae614-3212-4841-898b-85a98ba12041'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:11.007724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] The token cache is being invalidate because project 73afbf716079452ba1deb906dbdce949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:13:11.008032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:11.008710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d15ae614-3212-4841-898b-85a98ba12041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:13:11.010794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1163/2299] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:13:10 2022] DELETE /identity/v3/projects/73afbf716079452ba1deb906dbdce949 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:13:17.815674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:17.816803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:17.894727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7f3f-e03d-4902-a2b3-7d2754d312b6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:17.912404 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:17.968807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:17.969460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:17.969746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:17.970018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:17.971889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:18.039393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:18.041010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:18.052762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1137/2300] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:13:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:13:21.002874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:28.046105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=48 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:43.096122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:43.097100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:43.097100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:43.128949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] MFA Rules not processed for user `3cad6d025da54eb29d09fd2a2888c844`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:43.131184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:43.142880 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1164/2301] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:13:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:13:44.951826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:44.953753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:44.993916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d1e9f-9854-49d4-9797-9e86e66b83ab service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:45.004306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fd632-dfc0-4e09-a43b-412d7bc1600d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:45.005446 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd632-dfc0-4e09-a43b-412d7bc1600d service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:45.005972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd632-dfc0-4e09-a43b-412d7bc1600d service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:45.006407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd632-dfc0-4e09-a43b-412d7bc1600d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:45.010760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cc8fd632-dfc0-4e09-a43b-412d7bc1600d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:45.143277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fd632-dfc0-4e09-a43b-412d7bc1600d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:45.146332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fd632-dfc0-4e09-a43b-412d7bc1600d service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:45.163856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1138/2302] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:13:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:13:53.120715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:55.155764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=28 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:57.560342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:57.577198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:57.645611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32402f97-e559-4178-8dc5-e26ea1437eb9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:57.698895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd989791-0fc5-448f-807e-ce4761183261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:57.700301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd989791-0fc5-448f-807e-ce4761183261 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:57.702029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd989791-0fc5-448f-807e-ce4761183261 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:57.702029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd989791-0fc5-448f-807e-ce4761183261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:57.711243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cd989791-0fc5-448f-807e-ce4761183261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:57.760528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd989791-0fc5-448f-807e-ce4761183261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:57.764854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd989791-0fc5-448f-807e-ce4761183261 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:57.799411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1165/2303] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:13:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:14:04.440598 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:04.441495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:04.441986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:04.481918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] MFA Rules not processed for user `5235207aa3b5477eb76215ebb4a43c98`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:04.489313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:04.602647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1139/2304] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:14:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:07.790265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:10.183435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319925b5-38e2-4344-a26b-27f925b7ae98 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:10.185194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319925b5-38e2-4344-a26b-27f925b7ae98 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:10.185693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319925b5-38e2-4344-a26b-27f925b7ae98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:10.250825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-319925b5-38e2-4344-a26b-27f925b7ae98 None None] MFA Rules not processed for user `ca4779abc6024a0f84f041ca7c058e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:10.256165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-319925b5-38e2-4344-a26b-27f925b7ae98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:10.339821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1166/2305] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:14:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:14.589130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:20.326737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=15 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:24.061772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:24.065910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:24.440146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0a0ba-03d2-4a91-a6ed-78d8546c3050 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:24.450914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:24.451992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:24.452481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:24.452951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:24.460327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:24.483497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:24.491852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:24.523071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1140/2306] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:34.512312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=63 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:34.579757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319925b5-38e2-4344-a26b-27f925b7ae98 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:34.623735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319925b5-38e2-4344-a26b-27f925b7ae98 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:34.631595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:34.632301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:34.632655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:34.632958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:34.634937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:34.722829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:34.724554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:34.735782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1167/2307] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:14:44.730555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=35 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:17.414630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:17.416693 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:17.575648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1815656-71cb-4325-8eac-f46d07ea2971 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:17.583542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:17.584584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:17.585069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:17.585569 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:17.592832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:17.763854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:17.766398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:17.806472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1141/2308] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:23.658673 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:23.661642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:23.691063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eace755-4c69-45ba-bb5a-1f5ac16934ba service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:23.705782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:23.706921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:23.707414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:23.707878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:23.712046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:23.755550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:23.755550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:23.774868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:23.784460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:23.810137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1168/2309] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:15:23.819692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc328475-7870-4f23-ada7-7fd513fdd51b service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:23.838207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:23.846009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:23.847097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:23.847954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] PATH_INFO: `/v3/users/3cad6d025da54eb29d09fd2a2888c844` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:23.852148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cad6d025da54eb29d09fd2a2888c844)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:23.854931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:23.969619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cad6d025da54eb29d09fd2a2888c844'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:23.982942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cad6d025da54eb29d09fd2a2888c844'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:23.986661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:24.013158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cad6d025da54eb29d09fd2a2888c844'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.014392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cad6d025da54eb29d09fd2a2888c844'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.029524 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a691ebae-096e-445d-8988-a0e738978e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cad6d025da54eb29d09fd2a2888c844'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.041955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1142/2310] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:15:23 2022] DELETE /identity/v3/users/3cad6d025da54eb29d09fd2a2888c844 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:24.377321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:24.418547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9bf52-e963-4a52-ba0e-c8f984307b04 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:24.542790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:24.543892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:24.546924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:24.552501 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] PATH_INFO: `/v3/projects/9bb55accaa9046d99253a42a3154fe1b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:24.554960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb55accaa9046d99253a42a3154fe1b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:24.558963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:24.667019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb55accaa9046d99253a42a3154fe1b', 'request_id': 'req-bc60536d-164f-4791-af3c-8ec272b21f6e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.681257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb55accaa9046d99253a42a3154fe1b', 'request_id': 'req-bc60536d-164f-4791-af3c-8ec272b21f6e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.696161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb55accaa9046d99253a42a3154fe1b', 'request_id': 'req-bc60536d-164f-4791-af3c-8ec272b21f6e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.717444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] The token cache is being invalidate because project 9bb55accaa9046d99253a42a3154fe1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:15:24.717908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.718909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bc60536d-164f-4791-af3c-8ec272b21f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:24.721645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1169/2311] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:15:24 2022] DELETE /identity/v3/projects/9bb55accaa9046d99253a42a3154fe1b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:24.730247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deacba3-90e0-4136-8d54-dd04fd186cf8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:24.732528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deacba3-90e0-4136-8d54-dd04fd186cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:24.732528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deacba3-90e0-4136-8d54-dd04fd186cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:24.845919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0deacba3-90e0-4136-8d54-dd04fd186cf8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:24.848384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0deacba3-90e0-4136-8d54-dd04fd186cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:24.967702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1143/2312] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:24.978410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a218fdd-3b80-4755-9258-cd0c4c1e6b57 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:24.979045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a218fdd-3b80-4755-9258-cd0c4c1e6b57 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:24.979498 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a218fdd-3b80-4755-9258-cd0c4c1e6b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.016631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3a218fdd-3b80-4755-9258-cd0c4c1e6b57 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:25.018527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3a218fdd-3b80-4755-9258-cd0c4c1e6b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:25.033366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1170/2313] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:25.039254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deacba3-90e0-4136-8d54-dd04fd186cf8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:25.069919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deacba3-90e0-4136-8d54-dd04fd186cf8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:25.079398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:25.080495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.080990 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.081536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.082860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:15:25.083413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:25.086461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:25.111252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1144/2314] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:15:25.118231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a218fdd-3b80-4755-9258-cd0c4c1e6b57 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:25.135551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a218fdd-3b80-4755-9258-cd0c4c1e6b57 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:25.143049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:25.143814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.144188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.144606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.145849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:25.149829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:25.195983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1171/2315] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:25 2022] POST /identity/v3/projects => generated 369 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:25.202949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:25.221888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4bbb0-8e81-45c8-b629-046d2e72f7d2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:25.318543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:25.319188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.319471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.319746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.320425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:25.322552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:25.360608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1145/2316] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:25 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:25.367945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:25.391712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ab45c-acd1-4924-9289-6b85946e27d9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:25.400703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:25.401652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.402100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.402514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.403704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:25.406858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:25.433366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1172/2317] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:25.435973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:25.452939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f766069-83ac-4eca-a04b-10c9550206f3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:25.461581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c136589-b9c6-4e98-85a7-f729d1464242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:25.463829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c136589-b9c6-4e98-85a7-f729d1464242 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.463829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c136589-b9c6-4e98-85a7-f729d1464242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.463829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c136589-b9c6-4e98-85a7-f729d1464242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.467541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c136589-b9c6-4e98-85a7-f729d1464242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:25.467907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c136589-b9c6-4e98-85a7-f729d1464242 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:25.489078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1146/2318] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:15:25.496202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:25.514483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cecca-58ef-4f8f-be9a-c7b5ecd66fb3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:25.522586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:25.526054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.526504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.526944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] PATH_INFO: `/v3/projects/076179e7c0bf423597aca0343ff08014/users/1090c257bb0347e3bdb8fd3dc1e14fe5/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.559566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=076179e7c0bf423597aca0343ff08014, user_id=1090c257bb0347e3bdb8fd3dc1e14fe5, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:25.564922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:25.600801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1173/2319] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:25 2022] PUT /identity/v3/projects/076179e7c0bf423597aca0343ff08014/users/1090c257bb0347e3bdb8fd3dc1e14fe5/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:25.625975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58972d-1ac2-43b8-9622-80699eed9e8f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.626547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58972d-1ac2-43b8-9622-80699eed9e8f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.627054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58972d-1ac2-43b8-9622-80699eed9e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.675711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:25.704073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0e58972d-1ac2-43b8-9622-80699eed9e8f None None] MFA Rules not processed for user `1090c257bb0347e3bdb8fd3dc1e14fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:25.705624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e58972d-1ac2-43b8-9622-80699eed9e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:25.846896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1147/2320] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:25.854321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:25.883371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:25.979455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2a2f7-5143-4d42-bec2-c538d5d359ef admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:25.984791 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:25.985490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:25.985851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:25.986261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:25.987040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:25.990386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:26.036005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1174/2321] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:25 2022] POST /identity/v3/projects => generated 371 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:26.041819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58972d-1ac2-43b8-9622-80699eed9e8f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:26.062785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58972d-1ac2-43b8-9622-80699eed9e8f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:26.178865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:26.179963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:26.180478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:26.181243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:26.182370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:26.186246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:26.228187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1148/2322] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:26 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:26.234847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:26.252292 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f21d2e-c1a0-4795-b4ca-69271b4150e6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:26.260719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:26.261758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:26.262192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:26.262626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:26.263696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:26.266704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:26.284724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1175/2323] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:26.292418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:26.307615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af32b72-f93f-4441-875f-2720ec4d71ef admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:26.313382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:26.314382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:26.314775 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:26.315154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] PATH_INFO: `/v3/projects/1f48f7bf383b448999c99ef69f6a766f/users/ed6703f1cb444afb9ea16557a1e85c8b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:26.352866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f48f7bf383b448999c99ef69f6a766f, user_id=ed6703f1cb444afb9ea16557a1e85c8b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:26.360488 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:26.390836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1149/2324] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:26 2022] PUT /identity/v3/projects/1f48f7bf383b448999c99ef69f6a766f/users/ed6703f1cb444afb9ea16557a1e85c8b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:26.397951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:26.415910 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2a0ad-a480-4b0b-9594-b53c4af6d2ad admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:26.532630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849f37c-213e-48ad-baa2-3334d6a7421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:26.533851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849f37c-213e-48ad-baa2-3334d6a7421d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:26.534346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849f37c-213e-48ad-baa2-3334d6a7421d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:26.535287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849f37c-213e-48ad-baa2-3334d6a7421d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:26.536935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849f37c-213e-48ad-baa2-3334d6a7421d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:26.540138 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849f37c-213e-48ad-baa2-3334d6a7421d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:26.565866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1176/2325] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:26.568210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:26.586537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5214724-fd8f-40ed-8d89-c55dc51813bc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:26.595927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:26.597241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:26.599848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:26.599848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] PATH_INFO: `/v3/projects/1f48f7bf383b448999c99ef69f6a766f/users/ed6703f1cb444afb9ea16557a1e85c8b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:26.603534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f48f7bf383b448999c99ef69f6a766f, user_id=ed6703f1cb444afb9ea16557a1e85c8b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:26.609393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:26.633621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1150/2326] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:26 2022] PUT /identity/v3/projects/1f48f7bf383b448999c99ef69f6a766f/users/ed6703f1cb444afb9ea16557a1e85c8b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:26.653757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6e99-2035-4bca-a7fa-428fb41de7a5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:26.654249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6e99-2035-4bca-a7fa-428fb41de7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:26.654726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6e99-2035-4bca-a7fa-428fb41de7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:26.701331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2dee6e99-2035-4bca-a7fa-428fb41de7a5 None None] MFA Rules not processed for user `ed6703f1cb444afb9ea16557a1e85c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:26.703208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2dee6e99-2035-4bca-a7fa-428fb41de7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:26.852946 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1177/2327] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:26.871240 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:26.874069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:26.996610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:27.050774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641daae-4e4d-40f9-9114-4ad685ce412f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:27.058571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dd0ce-cfb6-44a0-b120-f5654686799e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:27.059636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dd0ce-cfb6-44a0-b120-f5654686799e service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:27.060137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dd0ce-cfb6-44a0-b120-f5654686799e service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:27.060624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dd0ce-cfb6-44a0-b120-f5654686799e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:27.079633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dd0ce-cfb6-44a0-b120-f5654686799e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:27.083989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dd0ce-cfb6-44a0-b120-f5654686799e service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:27.108898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1151/2328] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:27.799143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6e99-2035-4bca-a7fa-428fb41de7a5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:27.818094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:27.936216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6e99-2035-4bca-a7fa-428fb41de7a5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:27.943789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:27.944639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:27.944960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:27.945524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] PATH_INFO: `/v3/users/1090c257bb0347e3bdb8fd3dc1e14fe5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:27.947537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1090c257bb0347e3bdb8fd3dc1e14fe5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:27.950796 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:28.054545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1090c257bb0347e3bdb8fd3dc1e14fe5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:28.078365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1090c257bb0347e3bdb8fd3dc1e14fe5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:28.092871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1090c257bb0347e3bdb8fd3dc1e14fe5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:28.093932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1090c257bb0347e3bdb8fd3dc1e14fe5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:28.108607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1090c257bb0347e3bdb8fd3dc1e14fe5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:28.120373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1178/2329] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:15:27 2022] DELETE /identity/v3/users/1090c257bb0347e3bdb8fd3dc1e14fe5 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:28.128568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:28.129529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:28.129912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:28.165649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:28.167989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:28.293989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1152/2330] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:28.308560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:28.310785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:28.423876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0466306-db9d-40af-b61d-f9bf4a6af79d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:28.429311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:28.430116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:28.430601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:28.431089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:28.443907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:28.445989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:28.464786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1179/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:15:28.975515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:28.979124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:29.008566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e8c9a-213c-4656-8599-fdb75db6a9ae None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:29.033331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:29.034503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:29.035057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:29.035556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] PATH_INFO: `/v3/projects/076179e7c0bf423597aca0343ff08014` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:29.037951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] RBAC: Authorizing `identity:delete_project(project_id=076179e7c0bf423597aca0343ff08014)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:29.042366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:29.152139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076179e7c0bf423597aca0343ff08014', 'request_id': 'req-7991db15-2db0-4ba7-a944-c539aeec645c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.166409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076179e7c0bf423597aca0343ff08014', 'request_id': 'req-7991db15-2db0-4ba7-a944-c539aeec645c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.182186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076179e7c0bf423597aca0343ff08014', 'request_id': 'req-7991db15-2db0-4ba7-a944-c539aeec645c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.209708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] The token cache is being invalidate because project 076179e7c0bf423597aca0343ff08014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:15:29.210068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.210797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.213037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1153/2332] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:15:28 2022] DELETE /identity/v3/projects/076179e7c0bf423597aca0343ff08014 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:29.217534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:29.230889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5d1d7-e569-4f75-93f9-e2961a8b4a49 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:29.285464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:29.345960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:29.347099 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:29.347647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:29.349095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] PATH_INFO: `/v3/users/ed6703f1cb444afb9ea16557a1e85c8b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:29.351382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6703f1cb444afb9ea16557a1e85c8b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:29.354063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:29.457065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6703f1cb444afb9ea16557a1e85c8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.477268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6703f1cb444afb9ea16557a1e85c8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.490979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6703f1cb444afb9ea16557a1e85c8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.492021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6703f1cb444afb9ea16557a1e85c8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.506693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc142602-31c5-4b02-bb38-ee738355e73a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6703f1cb444afb9ea16557a1e85c8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:29.515587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1180/2333] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:15:29 2022] DELETE /identity/v3/users/ed6703f1cb444afb9ea16557a1e85c8b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:15:30.072121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:30.074580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:30.098701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991db15-2db0-4ba7-a944-c539aeec645c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:30.206595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:30.217172 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:30.221464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:30.221975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:30.222902 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] PATH_INFO: `/v3/projects/1f48f7bf383b448999c99ef69f6a766f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:30.225367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f48f7bf383b448999c99ef69f6a766f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:30.233499 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:30.319817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f48f7bf383b448999c99ef69f6a766f', 'request_id': 'req-56831b38-9d27-4f30-abe4-b351ea4541b1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:30.332449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f48f7bf383b448999c99ef69f6a766f', 'request_id': 'req-56831b38-9d27-4f30-abe4-b351ea4541b1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:30.345112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f48f7bf383b448999c99ef69f6a766f', 'request_id': 'req-56831b38-9d27-4f30-abe4-b351ea4541b1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:30.371319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] The token cache is being invalidate because project 1f48f7bf383b448999c99ef69f6a766f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:15:30.371827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:30.373056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-56831b38-9d27-4f30-abe4-b351ea4541b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:30.376218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1154/2334] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:15:30 2022] DELETE /identity/v3/projects/1f48f7bf383b448999c99ef69f6a766f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:30.387915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e69e6-b97d-475e-b631-8d20fe4bc9e8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:30.388635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e69e6-b97d-475e-b631-8d20fe4bc9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:30.388816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e69e6-b97d-475e-b631-8d20fe4bc9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:30.426215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a69e69e6-b97d-475e-b631-8d20fe4bc9e8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:30.428273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a69e69e6-b97d-475e-b631-8d20fe4bc9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:30.548503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1181/2335] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:30.561018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d4100-60c7-43e7-af8d-bf0d7911a960 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:30.561348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d4100-60c7-43e7-af8d-bf0d7911a960 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:30.561694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d4100-60c7-43e7-af8d-bf0d7911a960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:30.588456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7f3d4100-60c7-43e7-af8d-bf0d7911a960 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:30.590698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7f3d4100-60c7-43e7-af8d-bf0d7911a960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:30.605385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1155/2336] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:15:30.612889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e69e6-b97d-475e-b631-8d20fe4bc9e8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:30.628515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e69e6-b97d-475e-b631-8d20fe4bc9e8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:30.634807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:30.635444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:30.635716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:30.635985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:30.636731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:15:30.637044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:30.638771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:30.662827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1182/2337] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:15:30.669400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d4100-60c7-43e7-af8d-bf0d7911a960 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:30.692622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d4100-60c7-43e7-af8d-bf0d7911a960 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:30.701108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:30.702292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:30.702785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:30.703288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:30.704513 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:30.708453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:30.763682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1156/2338] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:30 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:30.770714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:30.788957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdd960-7563-4dc5-8173-ca33c4588f7c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:30.853061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:30.897696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:30.898792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:30.899285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:30.899767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:30.900959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:30.904820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:30.943274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1183/2339] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:30 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:30.950570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:30.971374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ec2ad-0207-4793-b010-5d7e400fe919 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:30.980352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:30.981139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:30.981547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:30.982020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:30.983095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:30.986239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:31.006895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1157/2340] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:15:31.013368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:31.031901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2056e-e67a-464a-a825-5564a39b6ab6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:31.040748 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcfd1a-442d-4e87-be2c-23f3bcad32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:31.041836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcfd1a-442d-4e87-be2c-23f3bcad32e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:31.042310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcfd1a-442d-4e87-be2c-23f3bcad32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:31.042755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcfd1a-442d-4e87-be2c-23f3bcad32e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:31.043897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcfd1a-442d-4e87-be2c-23f3bcad32e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:31.059782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcfd1a-442d-4e87-be2c-23f3bcad32e5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:31.094586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1184/2341] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:31.096149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:31.111939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e83937-c8a2-45cf-8d07-282075b43875 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:31.120382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:31.121206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:31.121570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:31.121948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] PATH_INFO: `/v3/projects/a19990c2a6354614bf9eb7e1cafa8102/users/628d29ce3c2440d8ac5ee527cd1464b0/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:31.155343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19990c2a6354614bf9eb7e1cafa8102, user_id=628d29ce3c2440d8ac5ee527cd1464b0, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:31.158664 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:31.183193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1158/2342] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:31 2022] PUT /identity/v3/projects/a19990c2a6354614bf9eb7e1cafa8102/users/628d29ce3c2440d8ac5ee527cd1464b0/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:15:31.202982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7a57-c3c6-44e8-8c64-37c560a4d000 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:31.203455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7a57-c3c6-44e8-8c64-37c560a4d000 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:31.203844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7a57-c3c6-44e8-8c64-37c560a4d000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:31.254590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d3fa7a57-c3c6-44e8-8c64-37c560a4d000 None None] MFA Rules not processed for user `628d29ce3c2440d8ac5ee527cd1464b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:31.256864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d3fa7a57-c3c6-44e8-8c64-37c560a4d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:31.369217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1185/2343] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:15:31.374717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:31.513435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c7e41-f442-4aa2-801f-ddc94d46e5e0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:31.519928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:31.520865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:31.521258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:31.521651 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:31.523393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:31.525661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:31.570788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1159/2344] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:31 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:31.576108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7a57-c3c6-44e8-8c64-37c560a4d000 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:31.593055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7a57-c3c6-44e8-8c64-37c560a4d000 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:31.701404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:31.702573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:31.703110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:31.703599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:31.704901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:31.708973 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:31.749999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1186/2345] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:31 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:31.756625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:31.777199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e8f2f-f766-4c3e-8496-59241837c49f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:31.784889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:31.786249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:31.786749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:31.787246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:31.788525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:31.791709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:31.796613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:31.822038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1160/2346] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:31.826042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:31.843793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa685e-4d05-41f5-9eae-43ebc2c30a0e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:31.853078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:31.853955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:31.854349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:31.854674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] PATH_INFO: `/v3/projects/790529e453274cdfbe0c705761a1c325/users/1285a82f883a47cc842e023bbd95a920/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:31.887087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790529e453274cdfbe0c705761a1c325, user_id=1285a82f883a47cc842e023bbd95a920, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:31.892325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:31.916016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1187/2347] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:31 2022] PUT /identity/v3/projects/790529e453274cdfbe0c705761a1c325/users/1285a82f883a47cc842e023bbd95a920/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:31.922655 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:31.941036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9a74-093b-4775-ad35-063158644a08 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:32.057215 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707739d-47e2-4dc8-8a9b-eb5892f9e7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:32.058274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707739d-47e2-4dc8-8a9b-eb5892f9e7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:32.058740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707739d-47e2-4dc8-8a9b-eb5892f9e7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:32.059209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707739d-47e2-4dc8-8a9b-eb5892f9e7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:32.060356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707739d-47e2-4dc8-8a9b-eb5892f9e7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:32.063300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707739d-47e2-4dc8-8a9b-eb5892f9e7b5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:32.083715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1161/2348] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:32.089407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:32.105036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfe53c-88d1-4663-9606-1ba68645fe30 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:32.110907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:32.111992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:32.112896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:32.113414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] PATH_INFO: `/v3/projects/790529e453274cdfbe0c705761a1c325/users/1285a82f883a47cc842e023bbd95a920/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:32.117953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790529e453274cdfbe0c705761a1c325, user_id=1285a82f883a47cc842e023bbd95a920, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:32.122581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:32.132995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:32.144762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1188/2349] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:32 2022] PUT /identity/v3/projects/790529e453274cdfbe0c705761a1c325/users/1285a82f883a47cc842e023bbd95a920/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:32.161991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ecf33-1d89-492b-a279-d004a636d8f6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:32.162479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ecf33-1d89-492b-a279-d004a636d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:32.162949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ecf33-1d89-492b-a279-d004a636d8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:32.208089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-053ecf33-1d89-492b-a279-d004a636d8f6 None None] MFA Rules not processed for user `1285a82f883a47cc842e023bbd95a920`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:32.209622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-053ecf33-1d89-492b-a279-d004a636d8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:32.334089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1162/2350] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:15:32.352807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:32.355236 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:32.474602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82e591-a4d7-401c-81c2-bf22117be2b5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:32.480494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:32.481457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:32.481878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:32.482230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:32.500433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:32.503426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:32.522013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1189/2351] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:32.654181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ecf33-1d89-492b-a279-d004a636d8f6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:32.672837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ecf33-1d89-492b-a279-d004a636d8f6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:32.680050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44677f8f-bea6-4015-9965-a7f425b54bba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:32.681122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677f8f-bea6-4015-9965-a7f425b54bba service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:32.681547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677f8f-bea6-4015-9965-a7f425b54bba service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:32.682082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677f8f-bea6-4015-9965-a7f425b54bba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:32.793045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44677f8f-bea6-4015-9965-a7f425b54bba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:32.796195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44677f8f-bea6-4015-9965-a7f425b54bba service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:32.818022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1163/2352] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:42.513882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=27 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:42.807324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:46.015340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:46.128181 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d51f6-a090-4cc0-96f4-58d173f28125 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:46.133530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:46.134440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:46.135725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:46.136027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] PATH_INFO: `/v3/users/628d29ce3c2440d8ac5ee527cd1464b0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:46.141534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628d29ce3c2440d8ac5ee527cd1464b0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:46.145639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:46.238240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628d29ce3c2440d8ac5ee527cd1464b0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:46.257786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628d29ce3c2440d8ac5ee527cd1464b0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:46.275659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628d29ce3c2440d8ac5ee527cd1464b0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:46.276893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628d29ce3c2440d8ac5ee527cd1464b0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:46.292275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628d29ce3c2440d8ac5ee527cd1464b0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:46.304750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1190/2353] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:15:46 2022] DELETE /identity/v3/users/628d29ce3c2440d8ac5ee527cd1464b0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:46.319018 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:46.319916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:46.319916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:46.359964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:46.362249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:46.477005 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1164/2354] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:46.490141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:46.492122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:46.601942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7fa92-a6a6-4c2d-9b7a-b8b7272f4fc6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:46.610396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:46.611396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:46.611795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:46.612254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:46.628613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:46.631180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:46.643688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1191/2355] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:47.153307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:47.155556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:47.182278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fe6fb-b4c3-42dd-914c-d12a8e913633 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:47.190952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:47.195367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:47.196066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:47.196629 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] PATH_INFO: `/v3/projects/a19990c2a6354614bf9eb7e1cafa8102` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:47.199293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19990c2a6354614bf9eb7e1cafa8102)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:47.203525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:47.300211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19990c2a6354614bf9eb7e1cafa8102', 'request_id': 'req-9d7acd54-3951-4b03-82d5-72007854c32a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.311883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19990c2a6354614bf9eb7e1cafa8102', 'request_id': 'req-9d7acd54-3951-4b03-82d5-72007854c32a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.324938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19990c2a6354614bf9eb7e1cafa8102', 'request_id': 'req-9d7acd54-3951-4b03-82d5-72007854c32a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.351382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] The token cache is being invalidate because project a19990c2a6354614bf9eb7e1cafa8102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:15:47.351777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.352636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.354894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1165/2356] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:15:47 2022] DELETE /identity/v3/projects/a19990c2a6354614bf9eb7e1cafa8102 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:47.359813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:47.373909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2444eb-791d-4ea1-8f51-d3dd122773fa service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:47.420019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:47.514548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:47.515674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:47.516153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:47.516633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] PATH_INFO: `/v3/users/1285a82f883a47cc842e023bbd95a920` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:47.519013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1285a82f883a47cc842e023bbd95a920)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:47.522864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:47.612682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1285a82f883a47cc842e023bbd95a920'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.636422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1285a82f883a47cc842e023bbd95a920'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.649352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1285a82f883a47cc842e023bbd95a920'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.650593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1285a82f883a47cc842e023bbd95a920'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.669747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2b1e9744-07d6-4ba4-9cca-190596e1f663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1285a82f883a47cc842e023bbd95a920'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:47.682598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1192/2357] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:15:47 2022] DELETE /identity/v3/users/1285a82f883a47cc842e023bbd95a920 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:15:48.224663 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:48.227367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:48.250582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7acd54-3951-4b03-82d5-72007854c32a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:48.377811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:48.380220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:48.381197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:48.382164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] PATH_INFO: `/v3/projects/790529e453274cdfbe0c705761a1c325` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:48.384586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] RBAC: Authorizing `identity:delete_project(project_id=790529e453274cdfbe0c705761a1c325)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:48.388587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:48.450956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:48.482695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790529e453274cdfbe0c705761a1c325', 'request_id': 'req-00453900-1a62-4324-86d3-83e397a998be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:48.491818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790529e453274cdfbe0c705761a1c325', 'request_id': 'req-00453900-1a62-4324-86d3-83e397a998be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:48.501560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790529e453274cdfbe0c705761a1c325', 'request_id': 'req-00453900-1a62-4324-86d3-83e397a998be'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:48.521400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] The token cache is being invalidate because project 790529e453274cdfbe0c705761a1c325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:15:48.521836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:48.522634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00453900-1a62-4324-86d3-83e397a998be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:48.524289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1166/2358] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:15:48 2022] DELETE /identity/v3/projects/790529e453274cdfbe0c705761a1c325 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:48.531133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80de343-1008-4777-928b-eb5e482d31e7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:48.531933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80de343-1008-4777-928b-eb5e482d31e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:48.531933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80de343-1008-4777-928b-eb5e482d31e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:48.559750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c80de343-1008-4777-928b-eb5e482d31e7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:48.561032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c80de343-1008-4777-928b-eb5e482d31e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:48.648044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1193/2359] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:15:48.658048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bb179-aab2-4381-8c75-98fb8d947adc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:48.658480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bb179-aab2-4381-8c75-98fb8d947adc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:48.659157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bb179-aab2-4381-8c75-98fb8d947adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:48.692807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-364bb179-aab2-4381-8c75-98fb8d947adc None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:48.695170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-364bb179-aab2-4381-8c75-98fb8d947adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:48.706949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1167/2360] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:48.711753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80de343-1008-4777-928b-eb5e482d31e7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:48.724495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80de343-1008-4777-928b-eb5e482d31e7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:48.730086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:48.730955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:48.731418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:48.731792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:48.732569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:15:48.733085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:48.735021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:48.750297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1194/2361] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:15:48.756618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bb179-aab2-4381-8c75-98fb8d947adc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:48.766757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bb179-aab2-4381-8c75-98fb8d947adc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:48.771937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:48.772588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:48.772844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:48.773113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:48.773804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:48.775802 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:48.819093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1168/2362] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:48 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:48.824380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:48.840473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f674f9-de9a-4b76-963c-e8069231799f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:48.884308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:48.930570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:48.931393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:48.931722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:48.932004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:48.932809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:48.937419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:48.969090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1195/2363] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:48 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:48.974900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:48.987068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853060a1-9c16-44ef-8775-e8ad2a80ca4b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:48.992639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:48.993234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:48.993516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:48.993829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:48.994500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:48.996522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.011594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1169/2364] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:49.017543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:49.031530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264106f7-daa3-491f-af18-32e11bf3b5b6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:49.039116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158a3fa-28e2-41d3-bd6e-266cf31d3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:49.040003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a3fa-28e2-41d3-bd6e-266cf31d3eac admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.040330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a3fa-28e2-41d3-bd6e-266cf31d3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.040649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a3fa-28e2-41d3-bd6e-266cf31d3eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.041417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158a3fa-28e2-41d3-bd6e-266cf31d3eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:49.044255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158a3fa-28e2-41d3-bd6e-266cf31d3eac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.060845 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1196/2365] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:15:49.067383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:49.081350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124ef9-db90-41ff-b1f9-5bbfec579d9c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:49.087935 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:49.089001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.089465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.089977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] PATH_INFO: `/v3/projects/1643cb88c57847ecb2b051257c56da64/users/5bcdcb7d092641a4a263a22131b1e33b/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.111591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1643cb88c57847ecb2b051257c56da64, user_id=5bcdcb7d092641a4a263a22131b1e33b, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:49.115140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.137386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1170/2366] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:49 2022] PUT /identity/v3/projects/1643cb88c57847ecb2b051257c56da64/users/5bcdcb7d092641a4a263a22131b1e33b/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:49.151938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e593-4de0-41e2-9655-0280ad7007fd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.152399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e593-4de0-41e2-9655-0280ad7007fd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.152936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e593-4de0-41e2-9655-0280ad7007fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.207679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d362e593-4de0-41e2-9655-0280ad7007fd None None] MFA Rules not processed for user `5bcdcb7d092641a4a263a22131b1e33b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:49.209861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d362e593-4de0-41e2-9655-0280ad7007fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:49.319891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1197/2367] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:49.326930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:49.443428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ad2d5-fb4f-4027-8251-a3070001ccb3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:49.451783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:49.453202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.453758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.454265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.455458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:49.459805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.507123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1171/2368] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:49 2022] POST /identity/v3/projects => generated 389 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:49.511734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e593-4de0-41e2-9655-0280ad7007fd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:49.524332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e593-4de0-41e2-9655-0280ad7007fd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:49.629008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:49.630390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.630588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.631030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.631713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:49.634457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.675686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1198/2369] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:49 2022] POST /identity/v3/users => generated 301 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:49.681857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:49.701828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431cfd5-8f9b-423c-878e-5994111352ee admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:49.710466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:49.711547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.712038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.712529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.713837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:49.717170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.737334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1172/2370] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:49.742415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:49.755132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8085741-e252-4dc1-a8ea-9ad083e6714f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:49.762531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:49.763397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.763682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.763961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] PATH_INFO: `/v3/projects/02c1d686150f46afa20a7392f4dfdde3/users/8a05ee8384384d6eb14af50101ec6a2e/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.786727 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c1d686150f46afa20a7392f4dfdde3, user_id=8a05ee8384384d6eb14af50101ec6a2e, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:49.790543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.816625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1199/2371] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:49 2022] PUT /identity/v3/projects/02c1d686150f46afa20a7392f4dfdde3/users/8a05ee8384384d6eb14af50101ec6a2e/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:15:49.823137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:49.846289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f9c12-5cf1-42e1-adbd-bbcb2b329121 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:49.888617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:49.970489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11839930-737b-4c84-abd6-7612fb0b2bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:49.971251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11839930-737b-4c84-abd6-7612fb0b2bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:49.971715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11839930-737b-4c84-abd6-7612fb0b2bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:49.971911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11839930-737b-4c84-abd6-7612fb0b2bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:49.972831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11839930-737b-4c84-abd6-7612fb0b2bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:49.975345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11839930-737b-4c84-abd6-7612fb0b2bb4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:49.992932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1173/2372] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:49.999587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:50.009197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:50.017984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ffb68-e93d-4733-8b17-ae9d2fc30745 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:50.027021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:50.027915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:50.028455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:50.032563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] PATH_INFO: `/v3/projects/02c1d686150f46afa20a7392f4dfdde3/users/8a05ee8384384d6eb14af50101ec6a2e/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:50.037432 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c1d686150f46afa20a7392f4dfdde3, user_id=8a05ee8384384d6eb14af50101ec6a2e, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:50.042649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:50.065409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1200/2373] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:49 2022] PUT /identity/v3/projects/02c1d686150f46afa20a7392f4dfdde3/users/8a05ee8384384d6eb14af50101ec6a2e/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:15:50.086206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e7f4-032d-4414-9e67-4734c537371d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:50.086681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e7f4-032d-4414-9e67-4734c537371d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:50.087147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e7f4-032d-4414-9e67-4734c537371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:50.140947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6ec0e7f4-032d-4414-9e67-4734c537371d None None] MFA Rules not processed for user `8a05ee8384384d6eb14af50101ec6a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:50.144557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6ec0e7f4-032d-4414-9e67-4734c537371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:50.248192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1174/2374] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:50.267176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:50.271671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:50.377570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e267c4-2714-4687-ba3b-802529ccd747 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:50.388451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c7973-5337-4be8-9044-c7caad99c81f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:50.389701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c7973-5337-4be8-9044-c7caad99c81f service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:50.390082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c7973-5337-4be8-9044-c7caad99c81f service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:50.390549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c7973-5337-4be8-9044-c7caad99c81f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:50.403898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c7973-5337-4be8-9044-c7caad99c81f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:50.405281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c7973-5337-4be8-9044-c7caad99c81f service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:50.418854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1201/2375] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:50 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:50.635858 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e7f4-032d-4414-9e67-4734c537371d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:50.759281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e7f4-032d-4414-9e67-4734c537371d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:50.766147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:50.767055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:50.767521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:50.767980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] PATH_INFO: `/v3/users/5bcdcb7d092641a4a263a22131b1e33b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:50.769328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcdcb7d092641a4a263a22131b1e33b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:50.772161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:50.861361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcdcb7d092641a4a263a22131b1e33b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:50.863891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:50.880814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcdcb7d092641a4a263a22131b1e33b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:50.904856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcdcb7d092641a4a263a22131b1e33b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:50.905832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcdcb7d092641a4a263a22131b1e33b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:50.917806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcdcb7d092641a4a263a22131b1e33b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:50.928032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1175/2376] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:15:50 2022] DELETE /identity/v3/users/5bcdcb7d092641a4a263a22131b1e33b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:50.933325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:50.934062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:50.934062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:50.960611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:50.961954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:51.051337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1202/2377] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:51.064027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:51.066152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:51.176311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfbf1a-c3c5-46a4-ac75-e9211fbc9825 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:51.183442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:51.184425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:51.184785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:51.185252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:51.202803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:51.205601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:51.225529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1176/2378] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:51.666702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:51.669120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:51.692509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fadff-2240-49f7-bc41-482f507df2cd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:51.701023 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:51.701941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:51.702399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:51.702870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] PATH_INFO: `/v3/projects/1643cb88c57847ecb2b051257c56da64` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:51.704939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1643cb88c57847ecb2b051257c56da64)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:51.709320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:51.773757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1643cb88c57847ecb2b051257c56da64', 'request_id': 'req-64f43034-3db3-4b1a-b0f5-7bf3f510d921'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:51.782358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1643cb88c57847ecb2b051257c56da64', 'request_id': 'req-64f43034-3db3-4b1a-b0f5-7bf3f510d921'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:51.792605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1643cb88c57847ecb2b051257c56da64', 'request_id': 'req-64f43034-3db3-4b1a-b0f5-7bf3f510d921'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:51.811046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] The token cache is being invalidate because project 1643cb88c57847ecb2b051257c56da64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:15:51.811393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:51.812243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:51.814156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1203/2379] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:15:51 2022] DELETE /identity/v3/projects/1643cb88c57847ecb2b051257c56da64 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:51.818918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:51.836800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02435f9-4735-42d0-adc1-dd70f7ec7502 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:51.916480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:51.917202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:51.917585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:51.918006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] PATH_INFO: `/v3/users/8a05ee8384384d6eb14af50101ec6a2e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:51.919449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a05ee8384384d6eb14af50101ec6a2e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:51.922009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:51.989166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a05ee8384384d6eb14af50101ec6a2e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:51.999597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a05ee8384384d6eb14af50101ec6a2e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.015247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:52.016992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a05ee8384384d6eb14af50101ec6a2e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.017898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a05ee8384384d6eb14af50101ec6a2e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.030427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68724b02-5f20-4960-8090-46101f4550c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a05ee8384384d6eb14af50101ec6a2e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.039143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1177/2380] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:15:51 2022] DELETE /identity/v3/users/8a05ee8384384d6eb14af50101ec6a2e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:15:52.438744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:52.441303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:52.458163 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f43034-3db3-4b1a-b0f5-7bf3f510d921 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:52.474365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:52.549343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:52.550167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:52.550523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:52.550802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] PATH_INFO: `/v3/projects/02c1d686150f46afa20a7392f4dfdde3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:52.553304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c1d686150f46afa20a7392f4dfdde3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:52.555456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:52.637721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c1d686150f46afa20a7392f4dfdde3', 'request_id': 'req-efdabb30-ffde-4c03-9e78-de04aa0e75b9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.650656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c1d686150f46afa20a7392f4dfdde3', 'request_id': 'req-efdabb30-ffde-4c03-9e78-de04aa0e75b9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.664656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c1d686150f46afa20a7392f4dfdde3', 'request_id': 'req-efdabb30-ffde-4c03-9e78-de04aa0e75b9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.680934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] The token cache is being invalidate because project 02c1d686150f46afa20a7392f4dfdde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:15:52.681261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.682261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-efdabb30-ffde-4c03-9e78-de04aa0e75b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:15:52.684283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1204/2381] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:15:52 2022] DELETE /identity/v3/projects/02c1d686150f46afa20a7392f4dfdde3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:52.693196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f0667-cad7-4e3d-985f-ba82f1974dde None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:52.693765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f0667-cad7-4e3d-985f-ba82f1974dde None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:52.694269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f0667-cad7-4e3d-985f-ba82f1974dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:52.730416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d22f0667-cad7-4e3d-985f-ba82f1974dde None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:52.732713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d22f0667-cad7-4e3d-985f-ba82f1974dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:52.813193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1178/2382] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:52.821960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831bfb9-c087-41cd-8a13-e5bfc769bae3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:52.822488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831bfb9-c087-41cd-8a13-e5bfc769bae3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:52.822817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831bfb9-c087-41cd-8a13-e5bfc769bae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:52.845200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4831bfb9-c087-41cd-8a13-e5bfc769bae3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:52.846556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4831bfb9-c087-41cd-8a13-e5bfc769bae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:52.856231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1205/2383] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:52.861063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f0667-cad7-4e3d-985f-ba82f1974dde None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:52.872850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f0667-cad7-4e3d-985f-ba82f1974dde None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:52.878313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:52.878985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:52.879244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:52.879536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:52.880289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:15:52.880591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:52.882380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:52.898413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1179/2384] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:15:52.903724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831bfb9-c087-41cd-8a13-e5bfc769bae3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:52.921576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831bfb9-c087-41cd-8a13-e5bfc769bae3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:52.930672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:52.931717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:52.932200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:52.932672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:52.933861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:52.937584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:52.979131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1206/2385] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:52 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:15:52.985911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.003245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fe263-65ab-4ae3-868a-fd66cb992b0b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.105487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.106571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.107044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.107527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.108907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.112781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.143347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1180/2386] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:52 2022] POST /identity/v3/users => generated 284 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:15:53.148171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.160374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf851874-3c78-43bf-9270-7072d0de38d9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.167022 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.167730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.168052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.168376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.169152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.170911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.184187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1207/2387] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:53.188951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.200594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf66ab-0981-4f9e-882b-653f9b7b7a8b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.208379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63fd88-b2d0-49b1-b300-b6d26ee446bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.209119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63fd88-b2d0-49b1-b300-b6d26ee446bd admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.209412 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63fd88-b2d0-49b1-b300-b6d26ee446bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.209715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63fd88-b2d0-49b1-b300-b6d26ee446bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.210793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63fd88-b2d0-49b1-b300-b6d26ee446bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.217491 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63fd88-b2d0-49b1-b300-b6d26ee446bd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.227804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1181/2388] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:15:53.232964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.248925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdc478-1705-44c9-a941-1b027255cefe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.262249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.264256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.265572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.266361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] PATH_INFO: `/v3/projects/7f2bab96d09642879aacc8a493280bf3/users/2235b6d986664e76824fcbd897c56ff4/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.316069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2bab96d09642879aacc8a493280bf3, user_id=2235b6d986664e76824fcbd897c56ff4, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.319459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.337767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1208/2389] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:53 2022] PUT /identity/v3/projects/7f2bab96d09642879aacc8a493280bf3/users/2235b6d986664e76824fcbd897c56ff4/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:53.349055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc937-6d91-4207-8503-cac3d00ed2e2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.349496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc937-6d91-4207-8503-cac3d00ed2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.349859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc937-6d91-4207-8503-cac3d00ed2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.383938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e17cc937-6d91-4207-8503-cac3d00ed2e2 None None] MFA Rules not processed for user `2235b6d986664e76824fcbd897c56ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:53.385218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e17cc937-6d91-4207-8503-cac3d00ed2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:53.461711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:53.495743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1182/2390] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:53.499879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.573938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:53.600841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f23d-96b4-4405-99cb-f2b4e48dec46 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.606826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.607514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.607838 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.608155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.608849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.611361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.648295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1209/2391] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:15:53 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:15:53.655672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cc937-6d91-4207-8503-cac3d00ed2e2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.672157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cc937-6d91-4207-8503-cac3d00ed2e2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.755485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.756031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.756316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.756593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.757263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.759284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.785151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1183/2392] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:15:53 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:15:53.790038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.806137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fd005-b1c1-4781-8b4f-49150826af55 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.815240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.816263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.816752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.818068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.819328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.822629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.836765 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1210/2393] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:15:53.842088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.853661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce880a-fedd-4e19-9418-d67030a36409 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.859508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.860343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.860673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.861071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] PATH_INFO: `/v3/projects/16900f443f994effbeb023ce1e4f416d/users/cf9d181c8930448bae7c23f545d017ff/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.880559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16900f443f994effbeb023ce1e4f416d, user_id=cf9d181c8930448bae7c23f545d017ff, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.883503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:53.901046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1184/2394] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:53 2022] PUT /identity/v3/projects/16900f443f994effbeb023ce1e4f416d/users/cf9d181c8930448bae7c23f545d017ff/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:15:53.905605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:53.917931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8da414-6ee8-4747-9d1b-43f33b6476dc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:53.990759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9b0f-ef78-4199-85fa-c02dea10f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:53.991397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9b0f-ef78-4199-85fa-c02dea10f2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:53.991677 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9b0f-ef78-4199-85fa-c02dea10f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:53.991949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9b0f-ef78-4199-85fa-c02dea10f2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:53.992612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce9b0f-ef78-4199-85fa-c02dea10f2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:53.994350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce9b0f-ef78-4199-85fa-c02dea10f2b9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:54.007212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1211/2395] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:15:54.012482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:54.035383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9f6ad-e6a7-4552-b5cc-cf520470fa5b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:54.041292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:54.042026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:54.042288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:54.042560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] PATH_INFO: `/v3/projects/16900f443f994effbeb023ce1e4f416d/users/cf9d181c8930448bae7c23f545d017ff/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:54.044782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16900f443f994effbeb023ce1e4f416d, user_id=cf9d181c8930448bae7c23f545d017ff, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:54.047853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:54.065725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1185/2396] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:15:54 2022] PUT /identity/v3/projects/16900f443f994effbeb023ce1e4f416d/users/cf9d181c8930448bae7c23f545d017ff/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:15:54.078924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8d405-0d80-4350-910d-ed97bcecbe9e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:54.079520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8d405-0d80-4350-910d-ed97bcecbe9e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:54.079792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8d405-0d80-4350-910d-ed97bcecbe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:54.121285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-98d8d405-0d80-4350-910d-ed97bcecbe9e None None] MFA Rules not processed for user `cf9d181c8930448bae7c23f545d017ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:15:54.122919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98d8d405-0d80-4350-910d-ed97bcecbe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:54.233048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1212/2397] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:15:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:15:54.250422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:54.254596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:15:54.391389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8a4c-681e-4c1a-a194-22181afc951a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:54.399926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:54.400811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:54.401241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:54.401720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:54.419925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:54.423169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:54.445225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1186/2398] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:15:54.592974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8d405-0d80-4350-910d-ed97bcecbe9e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:15:54.609920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8d405-0d80-4350-910d-ed97bcecbe9e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:15:54.616958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538abe1f-a736-4b9f-aed8-4c6ebd0a3f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:15:54.618064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538abe1f-a736-4b9f-aed8-4c6ebd0a3f0a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:15:54.618490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538abe1f-a736-4b9f-aed8-4c6ebd0a3f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:15:54.618926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538abe1f-a736-4b9f-aed8-4c6ebd0a3f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:15:54.686824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:15:54.726484 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538abe1f-a736-4b9f-aed8-4c6ebd0a3f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:15:54.728028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538abe1f-a736-4b9f-aed8-4c6ebd0a3f0a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:15:54.740502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1213/2399] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:15:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:00.368151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:00.445520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:00.487515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30a4ff-862c-4491-8d26-ad23b47f4257 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:00.498375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:00.499452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:00.500501 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:00.500993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] PATH_INFO: `/v3/users/2235b6d986664e76824fcbd897c56ff4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:00.502781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2235b6d986664e76824fcbd897c56ff4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:00.505246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:00.591244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2235b6d986664e76824fcbd897c56ff4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:00.607987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2235b6d986664e76824fcbd897c56ff4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:00.629088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2235b6d986664e76824fcbd897c56ff4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:00.630150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2235b6d986664e76824fcbd897c56ff4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:00.646219 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2235b6d986664e76824fcbd897c56ff4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:00.658695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1187/2400] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:00 2022] DELETE /identity/v3/users/2235b6d986664e76824fcbd897c56ff4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:00.666601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:00.667490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:00.667490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:00.699612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:00.700969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:00.813159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1214/2401] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:16:00.826580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:00.828539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:00.926091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f0d02-5cce-4027-ac18-99289fdc822a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:00.935378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:00.936461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:00.936943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:00.937407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:00.953141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:00.955099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:00.977853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1188/2402] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:01.425858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:01.428052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:01.450360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d4690-6647-4060-b132-cf72e39faa0f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:01.459124 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:01.460222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:01.460702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:01.461535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] PATH_INFO: `/v3/projects/7f2bab96d09642879aacc8a493280bf3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:01.463713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f2bab96d09642879aacc8a493280bf3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:01.467349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:01.570433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f2bab96d09642879aacc8a493280bf3', 'request_id': 'req-a6213143-a8b3-4cbd-86d0-e9add1288873'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.582427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f2bab96d09642879aacc8a493280bf3', 'request_id': 'req-a6213143-a8b3-4cbd-86d0-e9add1288873'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.596744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f2bab96d09642879aacc8a493280bf3', 'request_id': 'req-a6213143-a8b3-4cbd-86d0-e9add1288873'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.611082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] The token cache is being invalidate because project 7f2bab96d09642879aacc8a493280bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:01.611543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.612338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.614137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1215/2403] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:01 2022] DELETE /identity/v3/projects/7f2bab96d09642879aacc8a493280bf3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:01.619872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:01.632891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17091609-6bbe-42e6-a874-e09bb5688443 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:01.734485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:01.735211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:01.735397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:01.735638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] PATH_INFO: `/v3/users/cf9d181c8930448bae7c23f545d017ff` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:01.737718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9d181c8930448bae7c23f545d017ff)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:01.741154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:01.744122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:01.830169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9d181c8930448bae7c23f545d017ff'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.844771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9d181c8930448bae7c23f545d017ff'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.862511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9d181c8930448bae7c23f545d017ff'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.863439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9d181c8930448bae7c23f545d017ff'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.872980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75da6823-3d09-4c3a-b804-7ea71d96248a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9d181c8930448bae7c23f545d017ff'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:01.881438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1189/2404] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:01 2022] DELETE /identity/v3/users/cf9d181c8930448bae7c23f545d017ff => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:16:02.313986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:02.315412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:02.335072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6213143-a8b3-4cbd-86d0-e9add1288873 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:02.439642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:02.440367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:02.440698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:02.441067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] PATH_INFO: `/v3/projects/16900f443f994effbeb023ce1e4f416d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:02.442306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16900f443f994effbeb023ce1e4f416d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:02.444733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:02.446632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:02.507772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16900f443f994effbeb023ce1e4f416d', 'request_id': 'req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:02.517958 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16900f443f994effbeb023ce1e4f416d', 'request_id': 'req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:02.528520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16900f443f994effbeb023ce1e4f416d', 'request_id': 'req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:02.541421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] The token cache is being invalidate because project 16900f443f994effbeb023ce1e4f416d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:02.541784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:02.542528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2b7d112-70a6-49e5-ab8f-e16759b67fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:02.544648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1216/2405] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:02 2022] DELETE /identity/v3/projects/16900f443f994effbeb023ce1e4f416d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:02.558683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb988d-9488-45e1-a75f-eb0683b73466 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:02.559224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb988d-9488-45e1-a75f-eb0683b73466 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:02.559507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb988d-9488-45e1-a75f-eb0683b73466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:02.597506 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-35bb988d-9488-45e1-a75f-eb0683b73466 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:02.599772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-35bb988d-9488-45e1-a75f-eb0683b73466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:02.703415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1190/2406] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:02.714255 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc0a3b-e1b4-48cc-9b15-7842a4181a4d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:02.714730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc0a3b-e1b4-48cc-9b15-7842a4181a4d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:02.715376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc0a3b-e1b4-48cc-9b15-7842a4181a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:02.740963 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6afc0a3b-e1b4-48cc-9b15-7842a4181a4d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:02.742621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6afc0a3b-e1b4-48cc-9b15-7842a4181a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:02.755504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1217/2407] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:02.761090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb988d-9488-45e1-a75f-eb0683b73466 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:02.774152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb988d-9488-45e1-a75f-eb0683b73466 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:02.785697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:02.786510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:02.786851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:02.787313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:02.788466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:02.789000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:02.791722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:02.809090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1191/2408] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:16:02.810785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc0a3b-e1b4-48cc-9b15-7842a4181a4d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:02.829630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc0a3b-e1b4-48cc-9b15-7842a4181a4d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:02.838180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:02.839680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:02.839897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:02.840288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:02.843834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:02.844974 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:02.904972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1218/2409] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:02 2022] POST /identity/v3/projects => generated 361 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:02.906527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:02.929092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731809e-050b-47dd-99e8-ad0735cde80e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:03.061476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:03.062456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:03.063336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:03.063821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:03.065359 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:03.069197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:03.091225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:03.119536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1192/2410] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:02 2022] POST /identity/v3/users => generated 281 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:03.123180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:03.147104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59679016-7dab-4434-b1e6-34aad810346e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:03.176553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:03.177588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:03.178061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:03.178511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:03.179685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:03.182824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:03.206374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1219/2411] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:03.214977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:03.235753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52c1d-8082-42fc-80fe-57df09f925d5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:03.245095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143fd95-1866-4cf1-a8bd-c72338213d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:03.246204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143fd95-1866-4cf1-a8bd-c72338213d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:03.246705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143fd95-1866-4cf1-a8bd-c72338213d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:03.247164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143fd95-1866-4cf1-a8bd-c72338213d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:03.248411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143fd95-1866-4cf1-a8bd-c72338213d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:03.251667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143fd95-1866-4cf1-a8bd-c72338213d6f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:03.275281 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1193/2412] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:03.281239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:03.296178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfd97c-1376-4a24-b4e1-38dbc72ace58 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:03.303523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:03.304679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:03.305297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:03.305841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] PATH_INFO: `/v3/projects/168fdeeb86a9482ea677716512034000/users/3ea3a4a7ade54cdebdcf5759c87f4a61/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:03.356876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168fdeeb86a9482ea677716512034000, user_id=3ea3a4a7ade54cdebdcf5759c87f4a61, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:03.362166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:03.390395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1220/2413] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:03 2022] PUT /identity/v3/projects/168fdeeb86a9482ea677716512034000/users/3ea3a4a7ade54cdebdcf5759c87f4a61/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:03.411082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1999-f27b-467e-ab7a-f2ba13530fa5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:03.411447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1999-f27b-467e-ab7a-f2ba13530fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:03.411896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1999-f27b-467e-ab7a-f2ba13530fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:03.469833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e1bf1999-f27b-467e-ab7a-f2ba13530fa5 None None] MFA Rules not processed for user `3ea3a4a7ade54cdebdcf5759c87f4a61`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:03.471306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e1bf1999-f27b-467e-ab7a-f2ba13530fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:03.572323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1194/2414] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:16:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:03.618798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:03.621294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:03.706045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09f0b5-b410-488d-8370-595b48c6f955 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:03.710986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf75be-9035-403b-a351-1d071c6697c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:03.711617 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf75be-9035-403b-a351-1d071c6697c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:03.711897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf75be-9035-403b-a351-1d071c6697c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:03.712170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf75be-9035-403b-a351-1d071c6697c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:03.726254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf75be-9035-403b-a351-1d071c6697c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:03.728723 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf75be-9035-403b-a351-1d071c6697c4 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:03.747100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1221/2415] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:03.772898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1999-f27b-467e-ab7a-f2ba13530fa5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:03.875705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1999-f27b-467e-ab7a-f2ba13530fa5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:03.883424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:03.884531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:03.885038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:03.885508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] PATH_INFO: `/v3/users/3ea3a4a7ade54cdebdcf5759c87f4a61` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:03.888564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea3a4a7ade54cdebdcf5759c87f4a61)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:03.895608 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:03.962793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea3a4a7ade54cdebdcf5759c87f4a61'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:03.976261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea3a4a7ade54cdebdcf5759c87f4a61'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:03.992610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea3a4a7ade54cdebdcf5759c87f4a61'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:03.993398 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea3a4a7ade54cdebdcf5759c87f4a61'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:04.003431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:04.007274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea3a4a7ade54cdebdcf5759c87f4a61'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:04.016840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1195/2416] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:03 2022] DELETE /identity/v3/users/3ea3a4a7ade54cdebdcf5759c87f4a61 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:04.022015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:04.022389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:04.022823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:04.047308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:04.048531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:04.074207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:04.131105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1222/2417] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:04.141080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:04.142972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:04.242738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad3f1b-ef0b-4dd9-a85e-97eeffb8e57f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:04.247998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:04.248900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:04.248900 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:04.249530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:04.263262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:04.264933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:04.276284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1196/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:04.722835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:04.725729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:04.739861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845b256-299e-4e41-b702-57d84bc989b9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:04.755097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:04.756418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:04.757214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:04.758271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] PATH_INFO: `/v3/projects/168fdeeb86a9482ea677716512034000` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:04.760568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=168fdeeb86a9482ea677716512034000)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:04.765415 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:04.882942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168fdeeb86a9482ea677716512034000', 'request_id': 'req-4096f70f-885d-482e-a204-6e6b19612aa1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:04.899147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168fdeeb86a9482ea677716512034000', 'request_id': 'req-4096f70f-885d-482e-a204-6e6b19612aa1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:04.909058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168fdeeb86a9482ea677716512034000', 'request_id': 'req-4096f70f-885d-482e-a204-6e6b19612aa1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:04.922244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] The token cache is being invalidate because project 168fdeeb86a9482ea677716512034000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:04.922244 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:04.922764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:04.924318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1223/2419] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:04 2022] DELETE /identity/v3/projects/168fdeeb86a9482ea677716512034000 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:14.271055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=29 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:14.917261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:15.260072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:15.262616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:15.286312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55649b08-264c-4e03-a021-6a8b12d993ff service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:15.426360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f316fb0-3eb2-4ca1-bbed-ce14e1c9dbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:15.427740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f316fb0-3eb2-4ca1-bbed-ce14e1c9dbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:15.428245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f316fb0-3eb2-4ca1-bbed-ce14e1c9dbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:15.428739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f316fb0-3eb2-4ca1-bbed-ce14e1c9dbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:15.432494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2f316fb0-3eb2-4ca1-bbed-ce14e1c9dbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:15.594896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f316fb0-3eb2-4ca1-bbed-ce14e1c9dbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:15.596865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f316fb0-3eb2-4ca1-bbed-ce14e1c9dbf5 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:15.614158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1197/2420] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:15.736454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:15.739284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:15.762804 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096f70f-885d-482e-a204-6e6b19612aa1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:15.770950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:15.773097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:15.774619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:15.775361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:15.781220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:15.809977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:15.813795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:15.839216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1224/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:18.908150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146175-db99-46c2-8c13-940593c9d32d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.908994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146175-db99-46c2-8c13-940593c9d32d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.909451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146175-db99-46c2-8c13-940593c9d32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.945300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-87146175-db99-46c2-8c13-940593c9d32d None None] MFA Rules not processed for user `30153f2f99424f29afaf718784c9f6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.947859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-87146175-db99-46c2-8c13-940593c9d32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:19.099008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1198/2422] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:19.811245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.828909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211c06c-0412-401a-8e09-9158dd44d38f service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.836108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.837564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.838429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.839172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.842922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:19.888025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.890058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.911893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1225/2423] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:25.038666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:25.039233 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:25.039702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:25.104202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] MFA Rules not processed for user `ca4779abc6024a0f84f041ca7c058e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:25.106237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:25.120852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:25.202737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1199/2424] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:16:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:25.888618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:25.891066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:25.905543 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c496aa4-43e5-4998-b944-3f90b8e771a2 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:25.911397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:25.912759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:25.912951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:25.913211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:25.916367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:25.933875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:25.936162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:25.952790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1226/2425] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:27.619534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:27.622097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:27.738922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ca32b-6e08-4216-aaaf-ebd50a3897fa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:27.745765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c018e-68dc-4310-af90-c48ae6b23ea8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:27.746628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c018e-68dc-4310-af90-c48ae6b23ea8 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:27.747087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c018e-68dc-4310-af90-c48ae6b23ea8 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:27.747427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c018e-68dc-4310-af90-c48ae6b23ea8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:27.749514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7e7c018e-68dc-4310-af90-c48ae6b23ea8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:27.763746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c018e-68dc-4310-af90-c48ae6b23ea8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:27.765719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c018e-68dc-4310-af90-c48ae6b23ea8 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:27.778439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1200/2426] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:35.945010 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=18 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:36.159924 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:36.161414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:36.184876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518a81e-68e1-4d44-a195-d7dd74ede99d service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:36.192003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:36.192674 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:36.194714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:36.194714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:36.208121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:36.443435 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:36.446062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:36.502794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1227/2427] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:37.772819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=50 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:38.302207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6da59-72d7-4c1e-999b-3e828e1cedae None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:38.302591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6da59-72d7-4c1e-999b-3e828e1cedae None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:38.302874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6da59-72d7-4c1e-999b-3e828e1cedae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:38.393934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dce6da59-72d7-4c1e-999b-3e828e1cedae None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:38.395266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dce6da59-72d7-4c1e-999b-3e828e1cedae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:38.498186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1201/2428] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:16:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:38.567905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:38.570637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:38.753245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6a05f-b933-4831-8eeb-3c65b80856b4 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:38.760365 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980da40f-863d-4d46-befe-bbffd582bfa8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:38.761376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980da40f-863d-4d46-befe-bbffd582bfa8 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:38.761376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980da40f-863d-4d46-befe-bbffd582bfa8 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:38.762298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980da40f-863d-4d46-befe-bbffd582bfa8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:38.777421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980da40f-863d-4d46-befe-bbffd582bfa8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:38.778231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980da40f-863d-4d46-befe-bbffd582bfa8 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:38.790854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1228/2429] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:16:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:16:43.767809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6da59-72d7-4c1e-999b-3e828e1cedae None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:43.784028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6da59-72d7-4c1e-999b-3e828e1cedae None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:43.790955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ab0b-0eff-4ee4-bb8c-f437d7e6386c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:43.792184 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ab0b-0eff-4ee4-bb8c-f437d7e6386c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:43.792674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ab0b-0eff-4ee4-bb8c-f437d7e6386c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:43.793149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ab0b-0eff-4ee4-bb8c-f437d7e6386c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:43.796813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f502ab0b-0eff-4ee4-bb8c-f437d7e6386c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:43.815646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502ab0b-0eff-4ee4-bb8c-f437d7e6386c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:43.816956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502ab0b-0eff-4ee4-bb8c-f437d7e6386c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:43.844166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1202/2430] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:48.784704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=82 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:53.824254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=50 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:00.291555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4f580-f86f-409e-8f60-f3e43a58c29a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.292495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4f580-f86f-409e-8f60-f3e43a58c29a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.292696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4f580-f86f-409e-8f60-f3e43a58c29a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.295438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1229/2431] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:17:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:17:00.305090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359efd0-cd43-44bd-8e5e-3d77668c16f6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.306044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359efd0-cd43-44bd-8e5e-3d77668c16f6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.306044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359efd0-cd43-44bd-8e5e-3d77668c16f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.385717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b359efd0-cd43-44bd-8e5e-3d77668c16f6 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:00.388232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b359efd0-cd43-44bd-8e5e-3d77668c16f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:00.406128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1203/2432] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:00.432146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4f580-f86f-409e-8f60-f3e43a58c29a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:00.435669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5ae4f580-f86f-409e-8f60-f3e43a58c29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:00.576369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4f580-f86f-409e-8f60-f3e43a58c29a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:00.581128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04534d-bc0c-4d61-a3cb-2a19a4473ee2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:00.581909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04534d-bc0c-4d61-a3cb-2a19a4473ee2 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.582304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04534d-bc0c-4d61-a3cb-2a19a4473ee2 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.582694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04534d-bc0c-4d61-a3cb-2a19a4473ee2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.603553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04534d-bc0c-4d61-a3cb-2a19a4473ee2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:00.606268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04534d-bc0c-4d61-a3cb-2a19a4473ee2 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:00.627833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1230/2433] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:00.753798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320d786-698d-44bc-866d-0ffb9b633560 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.754129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320d786-698d-44bc-866d-0ffb9b633560 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.754561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320d786-698d-44bc-866d-0ffb9b633560 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.756287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1204/2434] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:17:00.764387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347555e3-8b5a-4ed7-8530-124c11458a64 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.764879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347555e3-8b5a-4ed7-8530-124c11458a64 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.765047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347555e3-8b5a-4ed7-8530-124c11458a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.830404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-347555e3-8b5a-4ed7-8530-124c11458a64 None None] MFA Rules not processed for user `4302574365514eb1b25bd8e62736a4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:00.832537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-347555e3-8b5a-4ed7-8530-124c11458a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:00.844366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1231/2435] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:00.862419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320d786-698d-44bc-866d-0ffb9b633560 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:00.877066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320d786-698d-44bc-866d-0ffb9b633560 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:00.889571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2a9d5-64ae-4bba-8b48-5c2a1105663e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:00.890498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2a9d5-64ae-4bba-8b48-5c2a1105663e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.890901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2a9d5-64ae-4bba-8b48-5c2a1105663e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.891229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2a9d5-64ae-4bba-8b48-5c2a1105663e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.909747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2a9d5-64ae-4bba-8b48-5c2a1105663e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:00.916950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2a9d5-64ae-4bba-8b48-5c2a1105663e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:00.933028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1205/2436] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:01.064862 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86b061-5417-4ec4-a8f4-7fef93ad66c0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:01.065317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86b061-5417-4ec4-a8f4-7fef93ad66c0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:01.065848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86b061-5417-4ec4-a8f4-7fef93ad66c0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:01.068083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1232/2437] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 13:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:17:01.076753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c216c38-7aff-4c96-ac08-adec00a73244 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:01.077547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c216c38-7aff-4c96-ac08-adec00a73244 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:01.077847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c216c38-7aff-4c96-ac08-adec00a73244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:01.108441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3c216c38-7aff-4c96-ac08-adec00a73244 None None] MFA Rules not processed for user `bd45335f7a65416a8e2535c582f0624a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:01.110871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3c216c38-7aff-4c96-ac08-adec00a73244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:01.127118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1206/2438] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 13:17:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:01.149502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86b061-5417-4ec4-a8f4-7fef93ad66c0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:01.171616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86b061-5417-4ec4-a8f4-7fef93ad66c0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:01.178208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:01.179248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:01.179719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:01.180189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:01.199754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:01.202636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:01.223942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1233/2439] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:04.075287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:04.075895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:04.075895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:04.099057 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:04.100752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:04.118551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1207/2440] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:17:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:04.136851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:04.156879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14185a-4469-42aa-aae1-e16f515df586 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:04.166177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b76691-c97a-4e15-8e52-d179f9f20ce3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:04.167201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b76691-c97a-4e15-8e52-d179f9f20ce3 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:04.167693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b76691-c97a-4e15-8e52-d179f9f20ce3 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:04.168152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b76691-c97a-4e15-8e52-d179f9f20ce3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:04.184593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b76691-c97a-4e15-8e52-d179f9f20ce3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:04.187610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b76691-c97a-4e15-8e52-d179f9f20ce3 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:04.204232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1234/2441] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:17:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:17:14.091647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=21 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:14.196750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=47 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:23.040552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:23.042916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:23.069764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3e3bd-b3f7-443d-ab79-95c80296d714 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:23.080173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:23.081160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:23.081632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:23.082122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:23.085126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:23.108727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:23.111980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:23.132417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1208/2442] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:25.663990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ed0b4-bace-4b28-bbc9-c3df68246109 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.664931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ed0b4-bace-4b28-bbc9-c3df68246109 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.664931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ed0b4-bace-4b28-bbc9-c3df68246109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.695000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a87ed0b4-bace-4b28-bbc9-c3df68246109 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:25.696232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a87ed0b4-bace-4b28-bbc9-c3df68246109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:25.706007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1235/2443] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:25.722522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:25.737408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b0a60-473f-4b39-adc6-be87bc00b89c service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:25.745921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:25.746514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.746793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.747061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.757820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:25.759410 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:25.769150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1209/2444] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:17:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:17:35.688049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:35.765416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=12 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:39.346439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ed0b4-bace-4b28-bbc9-c3df68246109 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:39.356991 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ed0b4-bace-4b28-bbc9-c3df68246109 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:39.360889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:39.361528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:39.361832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:39.362059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:39.363971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:39.449761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:39.451260 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:39.462262 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1236/2445] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:43.888702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.905447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a900609-a38f-4ca3-8398-f8c48519ff75 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.913976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.914994 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.915742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.916445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.919129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:43.958722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.961724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.985618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1210/2446] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:44.032296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.035850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.076152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66747-4eba-439d-86fa-7788387f7548 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.085599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.086509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.087033 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.087531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.090860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.125546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.129089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.155405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1237/2447] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:44.197631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.200517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.220492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634a977-4f74-45e5-b0ef-de6b6e098f0d service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.227399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.228422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.228961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.229477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] PATH_INFO: `/v3/users/ca4779abc6024a0f84f041ca7c058e9a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.231566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca4779abc6024a0f84f041ca7c058e9a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.240421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.349630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca4779abc6024a0f84f041ca7c058e9a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.360319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca4779abc6024a0f84f041ca7c058e9a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.384687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca4779abc6024a0f84f041ca7c058e9a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.385477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca4779abc6024a0f84f041ca7c058e9a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.395634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2db6ced9-82fb-434b-9e45-109bf12ea366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca4779abc6024a0f84f041ca7c058e9a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.406548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1211/2448] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:44 2022] DELETE /identity/v3/users/ca4779abc6024a0f84f041ca7c058e9a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:44.683552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.707199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72634d2-e9a4-4c31-8ecf-777d0e4fb1c4 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.849566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.850388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.850664 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.851111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] PATH_INFO: `/v3/projects/317ea4e7108a4fc099da6d76a1793ac9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.852518 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=317ea4e7108a4fc099da6d76a1793ac9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.854544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.921265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317ea4e7108a4fc099da6d76a1793ac9', 'request_id': 'req-26c3f0b4-be75-4a70-a4ec-453b0511e016'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.938249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317ea4e7108a4fc099da6d76a1793ac9', 'request_id': 'req-26c3f0b4-be75-4a70-a4ec-453b0511e016'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.948096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317ea4e7108a4fc099da6d76a1793ac9', 'request_id': 'req-26c3f0b4-be75-4a70-a4ec-453b0511e016'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.961702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] The token cache is being invalidate because project 317ea4e7108a4fc099da6d76a1793ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:44.961984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.963037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-26c3f0b4-be75-4a70-a4ec-453b0511e016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.965330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1238/2449] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:44 2022] DELETE /identity/v3/projects/317ea4e7108a4fc099da6d76a1793ac9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:44.973106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558dc54-f311-4d09-b245-da83b9b00825 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.974554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558dc54-f311-4d09-b245-da83b9b00825 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.974554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558dc54-f311-4d09-b245-da83b9b00825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.002876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4558dc54-f311-4d09-b245-da83b9b00825 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:45.004173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4558dc54-f311-4d09-b245-da83b9b00825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:45.118681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1212/2450] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:45.128395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a6f8-513c-450a-b818-eee435d4a43f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.128981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a6f8-513c-450a-b818-eee435d4a43f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.129316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a6f8-513c-450a-b818-eee435d4a43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.168156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d8e8a6f8-513c-450a-b818-eee435d4a43f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:45.169438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d8e8a6f8-513c-450a-b818-eee435d4a43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:45.179569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1239/2451] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:45.184430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558dc54-f311-4d09-b245-da83b9b00825 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.197245 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558dc54-f311-4d09-b245-da83b9b00825 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.202589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.203197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.203508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.203732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.204386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:45.204685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.206526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:45.227218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1213/2452] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:45.233632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a6f8-513c-450a-b818-eee435d4a43f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.240100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:45.247830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a6f8-513c-450a-b818-eee435d4a43f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.258834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.259831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.260328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.260820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.262161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.264491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:45.312557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1240/2453] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:45 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:45.319489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.337997 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc04e3-487c-4f4b-a896-03705e17ffeb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.430103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:45.461974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.463418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.463955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.464803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.466416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.470533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:45.513896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1214/2454] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:45 2022] POST /identity/v3/users => generated 284 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:45.515628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.536199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edde504-6c35-401e-a0c9-7e26e6afa033 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.543927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.545628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.546409 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.546907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.548114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.550986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:45.572171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1241/2455] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:45.577828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.591917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493760b7-8682-4f03-8fd4-524081c905d9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.597035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135eaf5-e270-4a13-84cc-2dbccdbf07d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.597797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135eaf5-e270-4a13-84cc-2dbccdbf07d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.598113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135eaf5-e270-4a13-84cc-2dbccdbf07d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.598397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135eaf5-e270-4a13-84cc-2dbccdbf07d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.599500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135eaf5-e270-4a13-84cc-2dbccdbf07d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.600687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135eaf5-e270-4a13-84cc-2dbccdbf07d0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:45.612704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1215/2456] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:45.617732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.636239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0b315-7636-47b9-8f48-e2dfbfee6297 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.642798 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.643734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.644452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.645098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] PATH_INFO: `/v3/projects/07163d7591114e9a9fc98ed2f73e7843/users/056d8dd987044067be5b8b9d5283f5c9/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.675905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07163d7591114e9a9fc98ed2f73e7843, user_id=056d8dd987044067be5b8b9d5283f5c9, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.679606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:45.696647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1242/2457] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:45 2022] PUT /identity/v3/projects/07163d7591114e9a9fc98ed2f73e7843/users/056d8dd987044067be5b8b9d5283f5c9/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:45.709188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.709498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.709815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.750258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] MFA Rules not processed for user `056d8dd987044067be5b8b9d5283f5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:45.752126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:45.850797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1216/2458] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:45.865034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.867303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:45.960577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a06c6b-b22d-4bc4-8f91-4b336bf0d1e6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.966705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.968178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.968178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.968655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.985311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.988105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:46.001319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1243/2459] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:49.291260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.293798 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:49.422356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f8f2-e7ae-4b04-af7d-729bbafef998 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.432849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.433895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.434429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.434814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] PATH_INFO: `/v3/users/5235207aa3b5477eb76215ebb4a43c98` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.436636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5235207aa3b5477eb76215ebb4a43c98)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.439138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.556540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5235207aa3b5477eb76215ebb4a43c98'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:49.570009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5235207aa3b5477eb76215ebb4a43c98'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:49.575341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:49.625863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5235207aa3b5477eb76215ebb4a43c98'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:49.627762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5235207aa3b5477eb76215ebb4a43c98'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:49.642343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1685b701-1c5f-480e-bc77-9ed08c0f05e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5235207aa3b5477eb76215ebb4a43c98'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:49.654792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1217/2460] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:49 2022] DELETE /identity/v3/users/5235207aa3b5477eb76215ebb4a43c98 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:50.266060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.280862 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cb195-ea0c-4b9a-8133-51a76be89ea8 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.391936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.392265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.392551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.392817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] PATH_INFO: `/v3/projects/f1959b8e00bc4819a993fdbd0c012f5c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.394232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1959b8e00bc4819a993fdbd0c012f5c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.396464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.463376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:50.469972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1959b8e00bc4819a993fdbd0c012f5c', 'request_id': 'req-477c42d3-4754-4a60-8a17-06673f93940c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:50.478479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1959b8e00bc4819a993fdbd0c012f5c', 'request_id': 'req-477c42d3-4754-4a60-8a17-06673f93940c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:50.489534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1959b8e00bc4819a993fdbd0c012f5c', 'request_id': 'req-477c42d3-4754-4a60-8a17-06673f93940c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:50.506217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] The token cache is being invalidate because project f1959b8e00bc4819a993fdbd0c012f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:50.506505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:50.507231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-477c42d3-4754-4a60-8a17-06673f93940c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:50.508777 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1244/2461] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:50 2022] DELETE /identity/v3/projects/f1959b8e00bc4819a993fdbd0c012f5c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:50.514151 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85210351-ac31-4e84-b6bf-a682bce2bc25 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.514945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85210351-ac31-4e84-b6bf-a682bce2bc25 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.514945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85210351-ac31-4e84-b6bf-a682bce2bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.541530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-85210351-ac31-4e84-b6bf-a682bce2bc25 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:50.543155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-85210351-ac31-4e84-b6bf-a682bce2bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:50.626507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1218/2462] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:50.636080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91510998-9a46-44c9-ab3b-0113321c807b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.636361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91510998-9a46-44c9-ab3b-0113321c807b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.636664 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91510998-9a46-44c9-ab3b-0113321c807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.662746 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-91510998-9a46-44c9-ab3b-0113321c807b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:50.664404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-91510998-9a46-44c9-ab3b-0113321c807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:50.675595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1245/2463] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:50.680159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85210351-ac31-4e84-b6bf-a682bce2bc25 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.690885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85210351-ac31-4e84-b6bf-a682bce2bc25 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.696143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.696767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.697050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.697326 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.698083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:50.698392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.700020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.714806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1219/2464] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:18:50.719462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91510998-9a46-44c9-ab3b-0113321c807b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.734861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91510998-9a46-44c9-ab3b-0113321c807b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.741037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.741729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.741977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.742276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.742891 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.745032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.784061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1246/2465] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:50 2022] POST /identity/v3/projects => generated 355 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:50.788453 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.799006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0123e4-1f79-40f8-bae8-3fca94e63d80 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.867765 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.868394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.868677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.868956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.869788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.871692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.899887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1220/2466] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:50 2022] POST /identity/v3/users => generated 278 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:50.906635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.922784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9ce09-d39c-4dc2-b28d-b8872db1ffaa admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.939073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.939977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.940391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.940792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.941908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.944541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.958850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1247/2467] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:50.965809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.984040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97a08e-19e7-4e2b-838d-03ad9da42ef8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.989757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b569eb5-a378-4828-8b48-101ed91bbbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.990403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569eb5-a378-4828-8b48-101ed91bbbeb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.990683 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569eb5-a378-4828-8b48-101ed91bbbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.990930 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569eb5-a378-4828-8b48-101ed91bbbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.991588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b569eb5-a378-4828-8b48-101ed91bbbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.993457 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b569eb5-a378-4828-8b48-101ed91bbbeb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:51.005139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1221/2468] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:51.009694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:51.020867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab413a-311f-4412-9aa4-dc82bc4f8500 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:51.026640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:51.027351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.027630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.027908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] PATH_INFO: `/v3/projects/1331a13e7eb6415cab88adcdf89a9ede/users/52b92bff8df442f6aab50011e090b810/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.048800 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1331a13e7eb6415cab88adcdf89a9ede, user_id=52b92bff8df442f6aab50011e090b810, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:51.052278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:51.068397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1248/2469] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:51 2022] PUT /identity/v3/projects/1331a13e7eb6415cab88adcdf89a9ede/users/52b92bff8df442f6aab50011e090b810/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:51.079640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd638-cc53-45e3-a42c-b34fd7d9c062 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.080119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd638-cc53-45e3-a42c-b34fd7d9c062 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.080622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd638-cc53-45e3-a42c-b34fd7d9c062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.124895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c0ccd638-cc53-45e3-a42c-b34fd7d9c062 None None] MFA Rules not processed for user `52b92bff8df442f6aab50011e090b810`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:51.126965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c0ccd638-cc53-45e3-a42c-b34fd7d9c062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:51.171733 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:51.217680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1222/2470] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:51.227048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:51.346941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2508d4d-fa93-4800-9e37-eeb3bdf9a8e7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:51.363234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:51.366024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.366757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.367316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.368523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:51.371576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:51.437836 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1249/2471] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:51 2022] POST /identity/v3/projects => generated 355 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:51.444059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccd638-cc53-45e3-a42c-b34fd7d9c062 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:51.462443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccd638-cc53-45e3-a42c-b34fd7d9c062 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:51.578852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:51.579897 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.580292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.580846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.582100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:51.586178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:51.643852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1223/2472] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:51 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:51.648588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:51.661646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179c51a-b94c-4a68-8914-74de1e9ac2a4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:51.669646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:51.670752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.671328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.672032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.676191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:51.676191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:51.708950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1250/2473] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:51.714621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:51.729460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae6d6-3086-4c49-a199-3a5dfcea1fab admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:51.736748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:51.737444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.737886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.738291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] PATH_INFO: `/v3/projects/c8285823f6bf4286af24068e7ebca4e0/users/59c9f3f17f574ca2a0b25bfe845b058b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.760304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8285823f6bf4286af24068e7ebca4e0, user_id=59c9f3f17f574ca2a0b25bfe845b058b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:51.764838 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:51.792547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1224/2474] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:51 2022] PUT /identity/v3/projects/c8285823f6bf4286af24068e7ebca4e0/users/59c9f3f17f574ca2a0b25bfe845b058b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:51.796859 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:51.811611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0681ecc-0bf9-435d-adc4-99fec9abb27a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:51.898507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:51.932132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01d86-202a-4b97-b2a4-c8a615b5efa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:51.933203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01d86-202a-4b97-b2a4-c8a615b5efa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.933706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01d86-202a-4b97-b2a4-c8a615b5efa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.934235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01d86-202a-4b97-b2a4-c8a615b5efa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.935271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01d86-202a-4b97-b2a4-c8a615b5efa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:51.939245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01d86-202a-4b97-b2a4-c8a615b5efa2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:51.963574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1251/2475] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:51.965273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:51.977607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008f677-a22f-4311-a0b2-c413d05b5ecb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:51.984950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:51.985646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:51.986016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:51.986295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] PATH_INFO: `/v3/projects/c8285823f6bf4286af24068e7ebca4e0/users/59c9f3f17f574ca2a0b25bfe845b058b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:51.991340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8285823f6bf4286af24068e7ebca4e0, user_id=59c9f3f17f574ca2a0b25bfe845b058b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:51.995401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:52.010081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1225/2476] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:51 2022] PUT /identity/v3/projects/c8285823f6bf4286af24068e7ebca4e0/users/59c9f3f17f574ca2a0b25bfe845b058b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:52.023631 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f81c-0007-4ddf-b04d-a4070caced61 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:52.024082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f81c-0007-4ddf-b04d-a4070caced61 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:52.024833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f81c-0007-4ddf-b04d-a4070caced61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:52.070952 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0448f81c-0007-4ddf-b04d-a4070caced61 None None] MFA Rules not processed for user `59c9f3f17f574ca2a0b25bfe845b058b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:52.073754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0448f81c-0007-4ddf-b04d-a4070caced61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:52.205003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1252/2477] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:52.222835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:52.226008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:52.339462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76859e09-e54f-49ea-9e86-e71fc6e5bd5c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:52.344271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:52.345236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:52.345657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:52.345908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:52.357352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:52.359029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:52.382626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1226/2478] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:52.546847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f81c-0007-4ddf-b04d-a4070caced61 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:52.564246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f81c-0007-4ddf-b04d-a4070caced61 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:52.570817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:52.572400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:52.573503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:52.573950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:52.670448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:52.672036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:52.687076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1253/2479] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:56.106104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:56.107576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:56.144448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:56.195183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76899b33-0c80-45b2-8370-96957fa13bf8 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:56.201604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac6e7b-428a-4c94-a4a3-a3b6a8ce3396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:56.202721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac6e7b-428a-4c94-a4a3-a3b6a8ce3396 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:56.203264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac6e7b-428a-4c94-a4a3-a3b6a8ce3396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:56.203760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac6e7b-428a-4c94-a4a3-a3b6a8ce3396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:56.206803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9aac6e7b-428a-4c94-a4a3-a3b6a8ce3396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:56.250247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:56.252003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:56.305578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac6e7b-428a-4c94-a4a3-a3b6a8ce3396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:56.307443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac6e7b-428a-4c94-a4a3-a3b6a8ce3396 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:56.321126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1227/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:56.356775 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5f058-2075-4a38-b814-ec35a32ee405 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:56.363139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:56.363908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:56.364180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:56.364683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] PATH_INFO: `/v3/users/056d8dd987044067be5b8b9d5283f5c9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:56.366342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=056d8dd987044067be5b8b9d5283f5c9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:56.368964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:56.372770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:56.457174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '056d8dd987044067be5b8b9d5283f5c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.476209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '056d8dd987044067be5b8b9d5283f5c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.495137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '056d8dd987044067be5b8b9d5283f5c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.496182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '056d8dd987044067be5b8b9d5283f5c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.508953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '056d8dd987044067be5b8b9d5283f5c9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.517161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1254/2481] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:56 2022] DELETE /identity/v3/users/056d8dd987044067be5b8b9d5283f5c9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:56.524262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:56.524626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:56.525381 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:56.550961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:56.553232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:56.660959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1228/2482] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:56.672557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:56.675412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:56.757978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46c3f2-daad-4d57-a4c9-7e6d3fadc59b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:56.762328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f1e84-928b-40c3-834c-265e9263fdab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:56.763217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f1e84-928b-40c3-834c-265e9263fdab service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:56.763822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f1e84-928b-40c3-834c-265e9263fdab service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:56.764560 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f1e84-928b-40c3-834c-265e9263fdab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:56.786057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f1e84-928b-40c3-834c-265e9263fdab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:56.788316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f1e84-928b-40c3-834c-265e9263fdab service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:56.807404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1255/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:57.247872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.251976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:57.275138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54ddcc-b673-4ce7-8b4b-c0c16ab0898e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.288149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.288821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.289639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.289969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] PATH_INFO: `/v3/projects/07163d7591114e9a9fc98ed2f73e7843` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.292910 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07163d7591114e9a9fc98ed2f73e7843)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.295135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.372211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07163d7591114e9a9fc98ed2f73e7843', 'request_id': 'req-f0cd44f5-cc11-49b8-8959-758ae35f3898'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.388486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07163d7591114e9a9fc98ed2f73e7843', 'request_id': 'req-f0cd44f5-cc11-49b8-8959-758ae35f3898'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.402819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07163d7591114e9a9fc98ed2f73e7843', 'request_id': 'req-f0cd44f5-cc11-49b8-8959-758ae35f3898'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.419192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:57.432419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] The token cache is being invalidate because project 07163d7591114e9a9fc98ed2f73e7843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:57.433015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.434364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f0cd44f5-cc11-49b8-8959-758ae35f3898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.436562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1229/2484] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:57 2022] DELETE /identity/v3/projects/07163d7591114e9a9fc98ed2f73e7843 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:57.444628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e337e-fd2a-4e4f-9235-848c1451afe0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.445866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e337e-fd2a-4e4f-9235-848c1451afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.445866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e337e-fd2a-4e4f-9235-848c1451afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.477624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4f9e337e-fd2a-4e4f-9235-848c1451afe0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:57.480173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f9e337e-fd2a-4e4f-9235-848c1451afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:57.596000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1256/2485] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:57.605715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28300921-026a-4bea-8391-d2e6371aea90 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.606180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28300921-026a-4bea-8391-d2e6371aea90 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.606633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28300921-026a-4bea-8391-d2e6371aea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.637951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-28300921-026a-4bea-8391-d2e6371aea90 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:57.639387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-28300921-026a-4bea-8391-d2e6371aea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:57.650416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1230/2486] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:57.656390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e337e-fd2a-4e4f-9235-848c1451afe0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.674710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e337e-fd2a-4e4f-9235-848c1451afe0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.683820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.684805 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.685246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.685790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.686846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:57.687351 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.690505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.713084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1257/2487] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:57.718417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28300921-026a-4bea-8391-d2e6371aea90 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.732333 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28300921-026a-4bea-8391-d2e6371aea90 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.737852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.738563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.738928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.739225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.740109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.742523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.789793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1231/2488] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:57.795679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.809884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fb08f-5411-42f5-810b-33823fe62dd7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.825060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:57.935061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.935882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.936360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.936931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.946751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.950807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.991216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1258/2489] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:57.996667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.017965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea81384-00f7-455a-b66e-4d68533facc4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.028384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.029342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.029880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.030367 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.032344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.034433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.050674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1232/2490] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:58.054941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.066659 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39f8cc-847e-4c96-8867-df8551492fc9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.071960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a8ee5-c06e-4dfd-90cc-12f025bdcf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.073640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a8ee5-c06e-4dfd-90cc-12f025bdcf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.073640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a8ee5-c06e-4dfd-90cc-12f025bdcf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.073640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a8ee5-c06e-4dfd-90cc-12f025bdcf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.075602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a8ee5-c06e-4dfd-90cc-12f025bdcf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.075602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a8ee5-c06e-4dfd-90cc-12f025bdcf29 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.089291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1259/2491] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:58.093597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.116115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bddb-6c35-41e9-a0ef-74f297aa7c3c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.129185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.130479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.131825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.131825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] PATH_INFO: `/v3/projects/ea70b7b14fc04f6ca91a0a4686ae9935/users/e99e0fe841d24d48bd7ddb21eee71708/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.176252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea70b7b14fc04f6ca91a0a4686ae9935, user_id=e99e0fe841d24d48bd7ddb21eee71708, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.182668 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.209591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1233/2492] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:58 2022] PUT /identity/v3/projects/ea70b7b14fc04f6ca91a0a4686ae9935/users/e99e0fe841d24d48bd7ddb21eee71708/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:58.230336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1af16-8383-4ee6-be76-bee53355120b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.230851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1af16-8383-4ee6-be76-bee53355120b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.231395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1af16-8383-4ee6-be76-bee53355120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.284278 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-12b1af16-8383-4ee6-be76-bee53355120b None None] MFA Rules not processed for user `e99e0fe841d24d48bd7ddb21eee71708`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:58.285825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12b1af16-8383-4ee6-be76-bee53355120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:58.417529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1260/2493] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:58.418898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.509931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c564b5b-48b5-407d-99ff-fd2a7b5ca1d4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.515867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.516527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.516845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.517085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.517807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.519777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.566790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1234/2494] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:58 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:58.575008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1af16-8383-4ee6-be76-bee53355120b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.599423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1af16-8383-4ee6-be76-bee53355120b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.733506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.737078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.738780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.738780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.740374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.745522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.808284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1261/2495] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:58 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:58.813822 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.843951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a844-b8c7-473c-8f55-6a15a3e46086 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.850932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.853725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.853725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.853725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.853725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.855960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.877471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1235/2496] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:58.881616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.893523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef999ea6-3493-4037-bf6f-bf36f9a2058c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.902753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.904534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.904979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.905443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] PATH_INFO: `/v3/projects/42e7b09ff4ec474bb96be43abc3e4a12/users/7b345e6e96ec44b182944f0a13fb3866/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.911646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:58.939570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e7b09ff4ec474bb96be43abc3e4a12, user_id=7b345e6e96ec44b182944f0a13fb3866, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.947472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.982583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1262/2497] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:58 2022] PUT /identity/v3/projects/42e7b09ff4ec474bb96be43abc3e4a12/users/7b345e6e96ec44b182944f0a13fb3866/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:58.987729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:59.015762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754dad-cf56-42d2-b57b-b828afe08eca admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:59.099378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77ae36-601c-4d10-8545-92b9e801e2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:59.100086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ae36-601c-4d10-8545-92b9e801e2df admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:59.100365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ae36-601c-4d10-8545-92b9e801e2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:59.100637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ae36-601c-4d10-8545-92b9e801e2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.101490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ae36-601c-4d10-8545-92b9e801e2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:59.103924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ae36-601c-4d10-8545-92b9e801e2df admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:59.128141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1236/2498] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:59.136457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:59.157734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fe09-1cc6-4c27-93db-f3daa7559f95 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:59.164728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:59.166006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:59.166288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:59.166563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] PATH_INFO: `/v3/projects/42e7b09ff4ec474bb96be43abc3e4a12/users/7b345e6e96ec44b182944f0a13fb3866/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.168812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e7b09ff4ec474bb96be43abc3e4a12, user_id=7b345e6e96ec44b182944f0a13fb3866, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:59.174583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:59.198294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1263/2499] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:59 2022] PUT /identity/v3/projects/42e7b09ff4ec474bb96be43abc3e4a12/users/7b345e6e96ec44b182944f0a13fb3866/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:59.210341 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469c024-611a-4fb6-bfac-c954b218a404 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:59.210815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469c024-611a-4fb6-bfac-c954b218a404 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:59.211294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469c024-611a-4fb6-bfac-c954b218a404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.224104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:59.263178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9469c024-611a-4fb6-bfac-c954b218a404 None None] MFA Rules not processed for user `7b345e6e96ec44b182944f0a13fb3866`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:59.264607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9469c024-611a-4fb6-bfac-c954b218a404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:59.403544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1237/2500] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:59.417447 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:59.419412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:59.531225 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b13c99-4265-4402-88ca-51a1c70f0acb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:59.535944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:59.536689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:59.537048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:59.537386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.550840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:59.552554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:59.572556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1264/2501] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:00.181461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469c024-611a-4fb6-bfac-c954b218a404 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.205760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469c024-611a-4fb6-bfac-c954b218a404 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.253173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b345e6e96ec44b182944f0a13fb3866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42e7b09ff4ec474bb96be43abc3e4a12', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.261380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.261772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.262503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.263605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:00.264142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.267275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.294714 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1238/2502] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:00.299713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.328494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22236f-50c2-4c33-bcce-8f331e4ddc57 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.344017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b345e6e96ec44b182944f0a13fb3866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42e7b09ff4ec474bb96be43abc3e4a12', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.345323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.346444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.347133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.348763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.352967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.420423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.428921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1265/2503] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/projects => generated 355 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:00.441817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30df37-936b-402c-8fbb-54cb1fe76621 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.601324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b345e6e96ec44b182944f0a13fb3866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42e7b09ff4ec474bb96be43abc3e4a12', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.604109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.605995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.606947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.610125 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:00.613134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.615736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.663864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1239/2504] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:19:00.769872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.798767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23f006-a6f1-48cb-8d13-b3d1cf8892b9 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.815912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b345e6e96ec44b182944f0a13fb3866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42e7b09ff4ec474bb96be43abc3e4a12', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.816677 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.816975 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.817251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] PATH_INFO: `/v3/projects/c4fefe1355cf4a2ba948a81b4b44ccc4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.818650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c4fefe1355cf4a2ba948a81b4b44ccc4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.820687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.931524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4fefe1355cf4a2ba948a81b4b44ccc4', 'request_id': 'req-6a240654-d78d-45ef-a6d3-5a295849a6a7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.943925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4fefe1355cf4a2ba948a81b4b44ccc4', 'request_id': 'req-6a240654-d78d-45ef-a6d3-5a295849a6a7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.966499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4fefe1355cf4a2ba948a81b4b44ccc4', 'request_id': 'req-6a240654-d78d-45ef-a6d3-5a295849a6a7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.991490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] The token cache is being invalidate because project c4fefe1355cf4a2ba948a81b4b44ccc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:00.991968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.993141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.995776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1266/2505] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:00 2022] DELETE /identity/v3/projects/c4fefe1355cf4a2ba948a81b4b44ccc4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:01.481111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:01.483591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:01.632711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23146-a78f-41d1-935b-8ea2029d1185 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.665156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:01.676376 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:01.679966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:01.745132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.746060 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.746351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.746689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] PATH_INFO: `/v3/users/e99e0fe841d24d48bd7ddb21eee71708` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.748028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:01.751156 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99e0fe841d24d48bd7ddb21eee71708)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.760325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.786623 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a240654-d78d-45ef-a6d3-5a295849a6a7 tempest-VolumeQuotaClassesTest-1830699733 tempest-VolumeQuotaClassesTest-1830699733-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.796550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.797227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.797751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.806870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] PATH_INFO: `/v3/users/30153f2f99424f29afaf718784c9f6b1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.838685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30153f2f99424f29afaf718784c9f6b1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.841490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.854048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99e0fe841d24d48bd7ddb21eee71708'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:01.868821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99e0fe841d24d48bd7ddb21eee71708'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:01.895241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99e0fe841d24d48bd7ddb21eee71708'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:01.896413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99e0fe841d24d48bd7ddb21eee71708'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:01.917953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-22e45e5d-abd1-4290-95ad-58b6c039b9b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99e0fe841d24d48bd7ddb21eee71708'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:01.939169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1240/2506] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:01 2022] DELETE /identity/v3/users/e99e0fe841d24d48bd7ddb21eee71708 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:01.946867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539e684-b633-46ea-a005-4f806169d289 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.947566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539e684-b633-46ea-a005-4f806169d289 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.948043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539e684-b633-46ea-a005-4f806169d289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.960786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30153f2f99424f29afaf718784c9f6b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:01.995607 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e539e684-b633-46ea-a005-4f806169d289 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:01.997504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e539e684-b633-46ea-a005-4f806169d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:02.000150 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30153f2f99424f29afaf718784c9f6b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.015849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30153f2f99424f29afaf718784c9f6b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.017063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30153f2f99424f29afaf718784c9f6b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.035230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30153f2f99424f29afaf718784c9f6b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.045540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1267/2507] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:01 2022] DELETE /identity/v3/users/30153f2f99424f29afaf718784c9f6b1 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:02.091357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1241/2508] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:02.105702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:02.110377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:02.235961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61f7a6-710e-4be2-9bd4-4a6c5dae4d2c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:02.242779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741c760-99ba-4609-8b53-696f62aaf120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:02.243787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c760-99ba-4609-8b53-696f62aaf120 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:02.244265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c760-99ba-4609-8b53-696f62aaf120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:02.244719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c760-99ba-4609-8b53-696f62aaf120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:02.363953 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539e684-b633-46ea-a005-4f806169d289 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:02.367352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e539e684-b633-46ea-a005-4f806169d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:02.389898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:02.404992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539e684-b633-46ea-a005-4f806169d289 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:02.405951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741c760-99ba-4609-8b53-696f62aaf120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:02.412240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741c760-99ba-4609-8b53-696f62aaf120 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:02.416299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:02.418001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:02.418528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:02.419097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] PATH_INFO: `/v3/projects/b33fcaf2f89d4438947508dee3737cee` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:02.440886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1268/2509] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:02.470939 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33fcaf2f89d4438947508dee3737cee)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:02.474627 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:02.555397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:02.603835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33fcaf2f89d4438947508dee3737cee', 'request_id': 'req-641a1bc8-03ee-4464-9cbb-0d240726e297'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.614944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33fcaf2f89d4438947508dee3737cee', 'request_id': 'req-641a1bc8-03ee-4464-9cbb-0d240726e297'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.628009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33fcaf2f89d4438947508dee3737cee', 'request_id': 'req-641a1bc8-03ee-4464-9cbb-0d240726e297'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.647860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] The token cache is being invalidate because project b33fcaf2f89d4438947508dee3737cee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:02.648463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.649401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-641a1bc8-03ee-4464-9cbb-0d240726e297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:02.651286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1242/2510] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:02 2022] DELETE /identity/v3/projects/b33fcaf2f89d4438947508dee3737cee => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:02.665185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246d59d-9a32-4ce7-9882-6c44a3f70cfb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:02.665961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246d59d-9a32-4ce7-9882-6c44a3f70cfb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:02.666668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246d59d-9a32-4ce7-9882-6c44a3f70cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:02.696383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6246d59d-9a32-4ce7-9882-6c44a3f70cfb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:02.696383 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6246d59d-9a32-4ce7-9882-6c44a3f70cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:02.831774 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1269/2511] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:02.842512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:02.842840 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:02.843168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:02.871605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:02.873008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:02.885036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1243/2512] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:02.890931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246d59d-9a32-4ce7-9882-6c44a3f70cfb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:02.913525 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246d59d-9a32-4ce7-9882-6c44a3f70cfb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:02.921325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:02.922418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:02.922895 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:02.923357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:02.924325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:02.924785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:02.927807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:02.947114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:02.949526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:02.971828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1270/2513] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:02.973199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:02.976048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081d443-d393-497e-9b52-2cfb636e36d0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:02.988026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2de99-6566-4d6a-a444-c0e2369b9afe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:02.988376 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:02.989380 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:02.989811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:02.990165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] PATH_INFO: `/v3/projects/ea70b7b14fc04f6ca91a0a4686ae9935` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:02.992173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea70b7b14fc04f6ca91a0a4686ae9935)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:02.994839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:03.010413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:03.011460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:03.011970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:03.012468 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:03.013569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:03.016243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:03.071369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1271/2514] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:02 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:03.075113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:03.088130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea70b7b14fc04f6ca91a0a4686ae9935', 'request_id': 'req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.095394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea70b7b14fc04f6ca91a0a4686ae9935', 'request_id': 'req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.103769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea70b7b14fc04f6ca91a0a4686ae9935', 'request_id': 'req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.132223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] The token cache is being invalidate because project ea70b7b14fc04f6ca91a0a4686ae9935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:03.132581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.133801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.135612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1244/2515] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:02 2022] DELETE /identity/v3/projects/ea70b7b14fc04f6ca91a0a4686ae9935 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:03.141426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:03.144766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:03.182787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb4766-a6f8-4bb8-ac5b-25c507f9962b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:03.267700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cae0e5-27cc-4a00-92d4-b4e7bfad4e22 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:03.276730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:03.277511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:03.277849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:03.278257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] PATH_INFO: `/v3/users/7b345e6e96ec44b182944f0a13fb3866` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:03.280667 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b345e6e96ec44b182944f0a13fb3866)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:03.282700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:03.287491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:03.288497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:03.288957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:03.289522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:03.290471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:03.293877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:03.310966 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:03.333815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1272/2516] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:03 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:03.341418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:03.343725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:03.345259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:03.362072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366e151-b537-4368-94b9-387f02549b5d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:03.368306 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:03.369050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:03.369361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:03.369704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b345e6e96ec44b182944f0a13fb3866'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.369839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:03.370604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:03.373076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:03.387666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b345e6e96ec44b182944f0a13fb3866'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.388732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1273/2517] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:03.394198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:03.408022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b345e6e96ec44b182944f0a13fb3866'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.409267 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a11955-79a1-4863-bf27-6d5944b232e5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:03.409437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b345e6e96ec44b182944f0a13fb3866'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.416784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad23770-677f-4ab3-8ca7-a4e3cafc6079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:03.418139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad23770-677f-4ab3-8ca7-a4e3cafc6079 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:03.418139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad23770-677f-4ab3-8ca7-a4e3cafc6079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:03.418913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad23770-677f-4ab3-8ca7-a4e3cafc6079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:03.420068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad23770-677f-4ab3-8ca7-a4e3cafc6079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:03.422534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b345e6e96ec44b182944f0a13fb3866'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:03.422902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad23770-677f-4ab3-8ca7-a4e3cafc6079 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:03.432043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1245/2518] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:03 2022] DELETE /identity/v3/users/7b345e6e96ec44b182944f0a13fb3866 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:03.445733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1274/2519] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:03.448662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:03.450915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:03.570409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be341-25da-4f62-8874-22dad7184a02 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:03.578405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:03.579492 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:03.579950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:03.580579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] PATH_INFO: `/v3/projects/1d8b76b1ffc14768882c073047ce26ea/users/005be735c503453ebf88cf6f3f57c22b/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:03.605538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d8b76b1ffc14768882c073047ce26ea, user_id=005be735c503453ebf88cf6f3f57c22b, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:03.609249 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:03.635522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1246/2520] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:03 2022] PUT /identity/v3/projects/1d8b76b1ffc14768882c073047ce26ea/users/005be735c503453ebf88cf6f3f57c22b/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:03.656793 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:03.657165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:03.657713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:03.723372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] MFA Rules not processed for user `005be735c503453ebf88cf6f3f57c22b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:03.725307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:03.853071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1275/2521] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:03.863022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:03.866230 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:03.922353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:03.924633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.023498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5d851-1bda-4554-8437-2b71befd769f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.034413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0657f27-da54-44a1-b8be-ee82c53f8824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.035556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0657f27-da54-44a1-b8be-ee82c53f8824 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.036424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0657f27-da54-44a1-b8be-ee82c53f8824 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.036842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0657f27-da54-44a1-b8be-ee82c53f8824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.051459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0657f27-da54-44a1-b8be-ee82c53f8824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.053981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0657f27-da54-44a1-b8be-ee82c53f8824 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.065700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbdf53-395a-4b41-aa2a-92d0823e3e43 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.071070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1247/2522] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:04.075056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.075624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.075977 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.076322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] PATH_INFO: `/v3/projects/42e7b09ff4ec474bb96be43abc3e4a12` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.077642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e7b09ff4ec474bb96be43abc3e4a12)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.080534 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.191416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e7b09ff4ec474bb96be43abc3e4a12', 'request_id': 'req-d08398a6-f986-4436-b1eb-82f7173d4295'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.205215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e7b09ff4ec474bb96be43abc3e4a12', 'request_id': 'req-d08398a6-f986-4436-b1eb-82f7173d4295'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.217594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e7b09ff4ec474bb96be43abc3e4a12', 'request_id': 'req-d08398a6-f986-4436-b1eb-82f7173d4295'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.227980 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:04.247018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] The token cache is being invalidate because project 42e7b09ff4ec474bb96be43abc3e4a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:04.247018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.247018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d08398a6-f986-4436-b1eb-82f7173d4295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.252427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1276/2523] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:03 2022] DELETE /identity/v3/projects/42e7b09ff4ec474bb96be43abc3e4a12 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:04.259922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a78c3e-d026-45a5-aa11-ff84b5b8c696 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.260212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a78c3e-d026-45a5-aa11-ff84b5b8c696 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.260523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a78c3e-d026-45a5-aa11-ff84b5b8c696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.285455 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-27a78c3e-d026-45a5-aa11-ff84b5b8c696 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:04.286849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-27a78c3e-d026-45a5-aa11-ff84b5b8c696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.300321 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:04.392049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1248/2524] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:04.400875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb2f24-871a-43ae-85ff-9497ba5f4814 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.401168 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb2f24-871a-43ae-85ff-9497ba5f4814 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.401397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb2f24-871a-43ae-85ff-9497ba5f4814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.432140 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cdbb2f24-871a-43ae-85ff-9497ba5f4814 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:04.433405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cdbb2f24-871a-43ae-85ff-9497ba5f4814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.445938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1277/2525] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:04.452643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a78c3e-d026-45a5-aa11-ff84b5b8c696 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.471337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a78c3e-d026-45a5-aa11-ff84b5b8c696 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.480428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.481659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.481972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.483365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.484503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:04.484859 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.487038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.510653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1249/2526] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:04.512669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb2f24-871a-43ae-85ff-9497ba5f4814 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.536422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb2f24-871a-43ae-85ff-9497ba5f4814 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.542783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.543907 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.544408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.544909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.546215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.548825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.597725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1278/2527] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:04 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:04.603293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.623520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd20d1c-4791-4941-8e64-4c089c409599 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.746781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.747814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.748086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.748372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.749589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.753709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.783723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1250/2528] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:04 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:04.789213 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.803564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221c7b-0b1a-4597-99fd-497ca06c8dda admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.809285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.809967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.810221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.810491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.811146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.812776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.824497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1279/2529] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:04.830657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.842025 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fd142-7885-467a-8ff8-2e463182c06a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.847435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c385683-444b-480f-aa19-eb65346aaae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.848067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385683-444b-480f-aa19-eb65346aaae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.848354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385683-444b-480f-aa19-eb65346aaae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.848632 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385683-444b-480f-aa19-eb65346aaae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.849342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c385683-444b-480f-aa19-eb65346aaae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.851038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c385683-444b-480f-aa19-eb65346aaae9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.868847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1251/2530] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:04.874102 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.891589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb2a4-de32-400d-8605-aad394935543 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.901882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.903460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.904008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.904512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] PATH_INFO: `/v3/projects/f47ce668d54145b8a589391b4e0092a7/users/83d4fc784bca4f618f6b16a226a22aa7/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.927391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47ce668d54145b8a589391b4e0092a7, user_id=83d4fc784bca4f618f6b16a226a22aa7, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.930964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.956458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1280/2531] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:04 2022] PUT /identity/v3/projects/f47ce668d54145b8a589391b4e0092a7/users/83d4fc784bca4f618f6b16a226a22aa7/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:04.964108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c4177-f9fc-4b4b-9824-8ff752d52ea7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.964548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c4177-f9fc-4b4b-9824-8ff752d52ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.964970 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c4177-f9fc-4b4b-9824-8ff752d52ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.987582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6e5c4177-f9fc-4b4b-9824-8ff752d52ea7 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:04.988941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6e5c4177-f9fc-4b4b-9824-8ff752d52ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:05.072112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1252/2532] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:05.083399 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:05.085228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:05.157788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44158c-d7b2-4df5-8efb-3caaa72d2103 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:05.161787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:05.162410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:05.162689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:05.162962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:05.178904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:05.181701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:05.200598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1281/2533] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:06.067306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56895-a856-4706-b163-9abe413d45c9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.067987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56895-a856-4706-b163-9abe413d45c9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.067987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a56895-a856-4706-b163-9abe413d45c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.110525 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b8a56895-a856-4706-b163-9abe413d45c9 None None] MFA Rules not processed for user `83d4fc784bca4f618f6b16a226a22aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:06.120619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b8a56895-a856-4706-b163-9abe413d45c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.134945 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:06.238810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1253/2534] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:06.240039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.251943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc597-c0df-473e-b5c2-1f78d03d31ba service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.262867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.263198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.263802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.264389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.265714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.269707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.305983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1282/2535] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:06.310470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a56895-a856-4706-b163-9abe413d45c9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.330255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a56895-a856-4706-b163-9abe413d45c9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.418589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.419630 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.420048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.420388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.421171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.423266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.451070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1254/2536] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:06.455455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.467241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af90da1-1c9c-4ffa-bd1a-098b24dd5872 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.472692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.473307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.473583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.473886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.474535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.476218 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.494392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1283/2537] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:06.499574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.512904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46092f54-4a3f-4b12-8901-a2aed6e4f43f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.519423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.520153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.520426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.520698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] PATH_INFO: `/v3/projects/278e22734c57426fa8c760ebddb039a5/users/0f24760ac5494571838ed1002efcda4e/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.542938 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] RBAC: Authorizing `identity:create_grant(project_id=278e22734c57426fa8c760ebddb039a5, user_id=0f24760ac5494571838ed1002efcda4e, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.545844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.568255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1255/2538] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:06 2022] PUT /identity/v3/projects/278e22734c57426fa8c760ebddb039a5/users/0f24760ac5494571838ed1002efcda4e/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:06.570335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.585981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4b9de-d2b0-41e5-ae4c-d4833e598b3f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.668100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.669519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.669782 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.670153 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.671320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.674670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.684118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:06.704087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1284/2539] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:06.705362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.717274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833788d5-57b6-4abf-a296-b3a24548f66f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.723464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55982ae-0185-4e06-8287-b63bec9f3a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.724159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55982ae-0185-4e06-8287-b63bec9f3a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.724431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55982ae-0185-4e06-8287-b63bec9f3a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.724694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55982ae-0185-4e06-8287-b63bec9f3a4c admin admin] PATH_INFO: `/v3/projects/278e22734c57426fa8c760ebddb039a5/users/0f24760ac5494571838ed1002efcda4e/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.727223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55982ae-0185-4e06-8287-b63bec9f3a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=278e22734c57426fa8c760ebddb039a5, user_id=0f24760ac5494571838ed1002efcda4e, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.730901 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55982ae-0185-4e06-8287-b63bec9f3a4c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.748225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1256/2540] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:06 2022] PUT /identity/v3/projects/278e22734c57426fa8c760ebddb039a5/users/0f24760ac5494571838ed1002efcda4e/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:07.265856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.265856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:07.289763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e657-7e20-4be4-81ea-275aaed436d1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.407156 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.408065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.408470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.408834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] PATH_INFO: `/v3/users/005be735c503453ebf88cf6f3f57c22b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.411000 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=005be735c503453ebf88cf6f3f57c22b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.413811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.496755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '005be735c503453ebf88cf6f3f57c22b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:07.514814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '005be735c503453ebf88cf6f3f57c22b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:07.539177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '005be735c503453ebf88cf6f3f57c22b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:07.541083 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '005be735c503453ebf88cf6f3f57c22b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:07.570967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '005be735c503453ebf88cf6f3f57c22b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:07.581873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1285/2541] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:07 2022] DELETE /identity/v3/users/005be735c503453ebf88cf6f3f57c22b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:07.589264 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b4782-1e55-4036-abf9-5cff8251c14a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.590243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b4782-1e55-4036-abf9-5cff8251c14a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.590958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b4782-1e55-4036-abf9-5cff8251c14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.626846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1d2b4782-1e55-4036-abf9-5cff8251c14a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:07.629158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d2b4782-1e55-4036-abf9-5cff8251c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:07.744646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1257/2542] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:07.756382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.758256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:07.864508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961981d-a59d-4358-bcf8-83f7e5526130 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.871077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.871908 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.872190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.872633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.893693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.896856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.920321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1286/2543] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:08.075673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.076193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.076715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.090987 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:08.128836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] MFA Rules not processed for user `0f24760ac5494571838ed1002efcda4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:08.130959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.314356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1258/2544] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:08.326940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.330243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.366597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:08.439682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff31c8-55ef-4375-9f22-73e5f9b4a6b5 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.446834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591549b8-6dfb-46e6-8eef-c66740213e04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.447629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591549b8-6dfb-46e6-8eef-c66740213e04 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.448037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591549b8-6dfb-46e6-8eef-c66740213e04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.448402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591549b8-6dfb-46e6-8eef-c66740213e04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.451487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-591549b8-6dfb-46e6-8eef-c66740213e04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.499061 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.500855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.522741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0c62d-b31d-42fb-81d8-e446a90f7175 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.534780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.536083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.536842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.537365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] PATH_INFO: `/v3/projects/1d8b76b1ffc14768882c073047ce26ea` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.540526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d8b76b1ffc14768882c073047ce26ea)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.544508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.553074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591549b8-6dfb-46e6-8eef-c66740213e04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.555288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591549b8-6dfb-46e6-8eef-c66740213e04 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.574393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1287/2545] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:08.670804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d8b76b1ffc14768882c073047ce26ea', 'request_id': 'req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:08.688009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d8b76b1ffc14768882c073047ce26ea', 'request_id': 'req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:08.699237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d8b76b1ffc14768882c073047ce26ea', 'request_id': 'req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:08.722841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] The token cache is being invalidate because project 1d8b76b1ffc14768882c073047ce26ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:08.723324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:08.724527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4d4fc1bd-e074-4aa6-899c-5e5afadd4e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:08.727246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1259/2546] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:08 2022] DELETE /identity/v3/projects/1d8b76b1ffc14768882c073047ce26ea => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:08.734113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea287e1d-5c88-4ae0-bf2d-48c4298ed1b0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.734588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea287e1d-5c88-4ae0-bf2d-48c4298ed1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.735038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea287e1d-5c88-4ae0-bf2d-48c4298ed1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.761201 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ea287e1d-5c88-4ae0-bf2d-48c4298ed1b0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:08.762644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ea287e1d-5c88-4ae0-bf2d-48c4298ed1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.847206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1288/2547] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:08.857174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29271f-801a-4c29-aac6-429d9081b8a9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.857597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29271f-801a-4c29-aac6-429d9081b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.858046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29271f-801a-4c29-aac6-429d9081b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.889204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4a29271f-801a-4c29-aac6-429d9081b8a9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:08.891192 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4a29271f-801a-4c29-aac6-429d9081b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.906126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1260/2548] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:08.912120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea287e1d-5c88-4ae0-bf2d-48c4298ed1b0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.924231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea287e1d-5c88-4ae0-bf2d-48c4298ed1b0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.932806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.933954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.934368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.934936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.936297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:08.936683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.939982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.967050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1289/2549] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:08.972350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29271f-801a-4c29-aac6-429d9081b8a9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.983781 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29271f-801a-4c29-aac6-429d9081b8a9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:09.002128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:09.003049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.003749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.004198 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.005362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:09.009707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:09.104161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1261/2550] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:08 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:09.105293 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:09.130164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3e674-af93-4dea-ac02-94e3d14dbb47 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:09.220007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:09.221050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.221657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.222357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.223897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:09.228849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:09.242507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:09.264453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1290/2551] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:09 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:09.268641 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:09.287425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf202d-763c-4c56-9ff1-100c373c6ae5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:09.293614 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:09.294273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.294552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.294823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.295485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:09.297113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:09.310036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1262/2552] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:09.314720 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:09.327730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec204d-25d7-417d-b7ae-e63941dfccc1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:09.333364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fe9a4-b491-4602-9160-24253aa6aa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:09.334032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe9a4-b491-4602-9160-24253aa6aa09 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.334309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe9a4-b491-4602-9160-24253aa6aa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.334588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe9a4-b491-4602-9160-24253aa6aa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.335273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fe9a4-b491-4602-9160-24253aa6aa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:09.337069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fe9a4-b491-4602-9160-24253aa6aa09 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:09.349454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1291/2553] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:09.354551 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:09.365006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878d75f-93d6-4137-afc2-a0219d246912 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:09.370701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:09.371419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.371695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.371969 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] PATH_INFO: `/v3/projects/8c2ed66d91784066b807d2ff34e9808d/users/6219ba9ad4264feebb8a40e5059ff060/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.394438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2ed66d91784066b807d2ff34e9808d, user_id=6219ba9ad4264feebb8a40e5059ff060, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:09.400916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:09.422783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1263/2554] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:09 2022] PUT /identity/v3/projects/8c2ed66d91784066b807d2ff34e9808d/users/6219ba9ad4264feebb8a40e5059ff060/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:09.435528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381aa97-0e3c-4e6a-a1d4-91e49b34cde0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.435528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381aa97-0e3c-4e6a-a1d4-91e49b34cde0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.435965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381aa97-0e3c-4e6a-a1d4-91e49b34cde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.467843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3381aa97-0e3c-4e6a-a1d4-91e49b34cde0 None None] MFA Rules not processed for user `6219ba9ad4264feebb8a40e5059ff060`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:09.469111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3381aa97-0e3c-4e6a-a1d4-91e49b34cde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:09.548929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1292/2555] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:09.561595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:09.563605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:09.576842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:09.650234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212827ff-6615-4fc3-be83-53990a723126 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:09.656609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:09.656777 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.657003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.657286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.666080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:09.667700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:09.678152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1264/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:09.998465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381aa97-0e3c-4e6a-a1d4-91e49b34cde0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:10.097841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381aa97-0e3c-4e6a-a1d4-91e49b34cde0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:10.103250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:10.103914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.104286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.104440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] PATH_INFO: `/v3/users/6219ba9ad4264feebb8a40e5059ff060` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.105794 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6219ba9ad4264feebb8a40e5059ff060)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:10.108092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:10.207604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6219ba9ad4264feebb8a40e5059ff060'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.208956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:10.210820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.223119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6219ba9ad4264feebb8a40e5059ff060'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.225029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd09005-83be-4810-9d59-257ab909517f service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:10.230451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:10.230767 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:10.231637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.232161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.232407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] PATH_INFO: `/v3/users/52b92bff8df442f6aab50011e090b810` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.235590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b92bff8df442f6aab50011e090b810)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:10.237886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:10.239007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6219ba9ad4264feebb8a40e5059ff060'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.239854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6219ba9ad4264feebb8a40e5059ff060'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.256026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6219ba9ad4264feebb8a40e5059ff060'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.269577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1293/2557] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:09 2022] DELETE /identity/v3/users/6219ba9ad4264feebb8a40e5059ff060 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:10.271154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:10.272179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.317542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b92bff8df442f6aab50011e090b810'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.328616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b92bff8df442f6aab50011e090b810'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.348297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b92bff8df442f6aab50011e090b810'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.349069 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b92bff8df442f6aab50011e090b810'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.358695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68d9c4cf-17bc-4cbe-a26d-2c78fb83ad6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b92bff8df442f6aab50011e090b810'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:10.362420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0b23a-bad7-44fb-a773-0c2135ff3153 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:10.366197 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1265/2558] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:10 2022] DELETE /identity/v3/users/52b92bff8df442f6aab50011e090b810 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:10.368338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:10.368817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332047a-4d97-420a-96b8-2d21f0f8d240 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.369111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332047a-4d97-420a-96b8-2d21f0f8d240 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.369392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332047a-4d97-420a-96b8-2d21f0f8d240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.369706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.370159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.370706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.373519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.395035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7332047a-4d97-420a-96b8-2d21f0f8d240 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:10.396445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7332047a-4d97-420a-96b8-2d21f0f8d240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.490433 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1266/2559] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:10.492639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.493128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.493420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.496563 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:10.500229 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:10.508962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:10.517020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1294/2560] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:10.518557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:10.518842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:10.519870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.520779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.533548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1267/2561] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:10.551032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:10.553445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.629941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61204ef1-ccaa-4e3d-a9e4-a9724a84e09e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:10.677780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec315e-42cb-4f42-b0d1-91be5d109253 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:10.684363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:10.685563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.686252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.686670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.700962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:10.703309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:10.717545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1268/2562] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:10.732458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:10.733955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.734421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.735027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.750957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:10.755394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:10.759203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:10.780188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1295/2563] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:11.207837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.209480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:11.222523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc9d7-be4a-4e5e-beeb-611c31a8b7c0 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.227421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.228074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.228342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.228616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] PATH_INFO: `/v3/projects/1331a13e7eb6415cab88adcdf89a9ede` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.229855 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1331a13e7eb6415cab88adcdf89a9ede)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.231866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.282350 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.284210 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:11.300562 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1331a13e7eb6415cab88adcdf89a9ede', 'request_id': 'req-551eb200-9138-4603-af14-5e276af9331f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.305167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94ec7f-cecb-45be-92ab-abcd2e36696a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.309349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1331a13e7eb6415cab88adcdf89a9ede', 'request_id': 'req-551eb200-9138-4603-af14-5e276af9331f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.318597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1331a13e7eb6415cab88adcdf89a9ede', 'request_id': 'req-551eb200-9138-4603-af14-5e276af9331f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.333932 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] The token cache is being invalidate because project 1331a13e7eb6415cab88adcdf89a9ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:11.334269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.334848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.345244 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1269/2564] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:11 2022] DELETE /identity/v3/projects/1331a13e7eb6415cab88adcdf89a9ede => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:11.346523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.348328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:11.376860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.377913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.377913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.378194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] PATH_INFO: `/v3/projects/8c2ed66d91784066b807d2ff34e9808d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.380066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2ed66d91784066b807d2ff34e9808d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.383810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.430715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551eb200-9138-4603-af14-5e276af9331f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.440474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2ed66d91784066b807d2ff34e9808d', 'request_id': 'req-e17edf74-cb39-41c9-9e84-9c09943aaf20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.448474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2ed66d91784066b807d2ff34e9808d', 'request_id': 'req-e17edf74-cb39-41c9-9e84-9c09943aaf20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.464886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2ed66d91784066b807d2ff34e9808d', 'request_id': 'req-e17edf74-cb39-41c9-9e84-9c09943aaf20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.467085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:11.484289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] The token cache is being invalidate because project 8c2ed66d91784066b807d2ff34e9808d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:11.484873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.485874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e17edf74-cb39-41c9-9e84-9c09943aaf20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.487466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1296/2565] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:11 2022] DELETE /identity/v3/projects/8c2ed66d91784066b807d2ff34e9808d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:11.492730 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a582-0e14-45ef-93a2-7cc894844f75 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.493353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a582-0e14-45ef-93a2-7cc894844f75 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.493353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a582-0e14-45ef-93a2-7cc894844f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.504968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.505710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.506013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.506282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] PATH_INFO: `/v3/users/59c9f3f17f574ca2a0b25bfe845b058b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.507883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c9f3f17f574ca2a0b25bfe845b058b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.510065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.513922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a875a582-0e14-45ef-93a2-7cc894844f75 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:11.515208 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a875a582-0e14-45ef-93a2-7cc894844f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:11.529217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1297/2566] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:11.539487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff413b0-e723-4b6f-a10f-de39884724dd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.540052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff413b0-e723-4b6f-a10f-de39884724dd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.540318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff413b0-e723-4b6f-a10f-de39884724dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.567868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2ff413b0-e723-4b6f-a10f-de39884724dd None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:11.569427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2ff413b0-e723-4b6f-a10f-de39884724dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:11.575609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c9f3f17f574ca2a0b25bfe845b058b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.581285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1298/2567] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:11.585537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff413b0-e723-4b6f-a10f-de39884724dd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.590532 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c9f3f17f574ca2a0b25bfe845b058b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.597828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff413b0-e723-4b6f-a10f-de39884724dd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.602852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.603487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.603751 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.604024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.604680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:11.604986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.606654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.606998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c9f3f17f574ca2a0b25bfe845b058b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.608115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c9f3f17f574ca2a0b25bfe845b058b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.617004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c9f3f17f574ca2a0b25bfe845b058b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:11.623474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1299/2568] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:11.623984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1270/2569] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:11 2022] DELETE /identity/v3/users/59c9f3f17f574ca2a0b25bfe845b058b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:11.631812 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.634609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:11.650937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad0ce-abb4-4246-a757-1d3f4fbf944b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.733132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.733820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.734076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.734348 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.735002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.737203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.743115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:11.777440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1300/2570] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:11 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:11.782706 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.798002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e62f-19f2-4f10-a065-fa4439c45c89 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.875921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.876531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.876814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.877087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.877759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.879697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.907665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1271/2571] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:11 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:11.913143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.934598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dc289-ac2a-4202-b478-672f0f1eab13 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.940943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.941783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.942049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.942314 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.943589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.945820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.968483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1301/2572] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:11.969584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.980690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07e893-5cc8-4d78-9203-19c6db59c772 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.987100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.988111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.988604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.989082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.990265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.993309 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.001170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.003848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:12.029657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1272/2573] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:12.031328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.035591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2571d6-4cc5-4452-81cf-cfc699cb0ad0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.043668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.044658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.045125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.045581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] PATH_INFO: `/v3/projects/c8285823f6bf4286af24068e7ebca4e0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.047647 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8285823f6bf4286af24068e7ebca4e0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.051212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.053028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bb557-da6b-4839-941d-6f953abd9ea7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.060056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cf6b5-8ed3-4ec1-9ec3-e2bb986d8e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.060746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cf6b5-8ed3-4ec1-9ec3-e2bb986d8e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.061109 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cf6b5-8ed3-4ec1-9ec3-e2bb986d8e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.061580 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cf6b5-8ed3-4ec1-9ec3-e2bb986d8e7e admin admin] PATH_INFO: `/v3/projects/718cdda592474987ab7afb3eb0576e0f/users/2ca42b910fc84bde9e82db1905fab6b4/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.079662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cf6b5-8ed3-4ec1-9ec3-e2bb986d8e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=718cdda592474987ab7afb3eb0576e0f, user_id=2ca42b910fc84bde9e82db1905fab6b4, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.082355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cf6b5-8ed3-4ec1-9ec3-e2bb986d8e7e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.099497 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1273/2574] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:12 2022] PUT /identity/v3/projects/718cdda592474987ab7afb3eb0576e0f/users/2ca42b910fc84bde9e82db1905fab6b4/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:12.117909 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.118265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.118564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.135335 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:12.136034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8285823f6bf4286af24068e7ebca4e0', 'request_id': 'req-60e877a7-65ec-4a64-b055-6bf0e37b2258'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:12.147209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8285823f6bf4286af24068e7ebca4e0', 'request_id': 'req-60e877a7-65ec-4a64-b055-6bf0e37b2258'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:12.160041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8285823f6bf4286af24068e7ebca4e0', 'request_id': 'req-60e877a7-65ec-4a64-b055-6bf0e37b2258'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:12.163555 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] MFA Rules not processed for user `2ca42b910fc84bde9e82db1905fab6b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:12.166408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:12.175263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] The token cache is being invalidate because project c8285823f6bf4286af24068e7ebca4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:12.175564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:12.176482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-60e877a7-65ec-4a64-b055-6bf0e37b2258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:12.178481 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1302/2575] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:11 2022] DELETE /identity/v3/projects/c8285823f6bf4286af24068e7ebca4e0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:12.183670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2521d6-3644-4068-a9c4-b31b4b7e1c93 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.183956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2521d6-3644-4068-a9c4-b31b4b7e1c93 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.184234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2521d6-3644-4068-a9c4-b31b4b7e1c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.208045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0c2521d6-3644-4068-a9c4-b31b4b7e1c93 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:12.209313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0c2521d6-3644-4068-a9c4-b31b4b7e1c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:12.279268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1274/2576] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:12.283633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.285128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:12.371508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1303/2577] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:12.380487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8877b0-7b7b-4d92-927e-054a4032e285 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.381477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8877b0-7b7b-4d92-927e-054a4032e285 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.381477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8877b0-7b7b-4d92-927e-054a4032e285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.407939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9b8877b0-7b7b-4d92-927e-054a4032e285 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:12.409323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9b8877b0-7b7b-4d92-927e-054a4032e285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:12.419750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1304/2578] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:12.421868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedb8cc-fd1b-4c1d-bed6-b8660443eb28 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.424654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8877b0-7b7b-4d92-927e-054a4032e285 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.426986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.427605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.427975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.428357 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.428988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.431871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.438120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8877b0-7b7b-4d92-927e-054a4032e285 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.443608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.444786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.444786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.445050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.445716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:12.446176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.448055 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.466987 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1305/2579] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:12.468915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.479789 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1275/2580] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:12 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:12.481814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12ac0b-2380-4ae1-8785-e81d0f721397 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.486223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.535009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:12.571547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.572209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.572671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.572834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.573466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.575655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.581698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227f866-d158-45a8-a1b4-9acaed20fa67 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.587672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.588077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.588356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.588754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.589265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.591350 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.619175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1276/2581] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:12 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:12.621508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1306/2582] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:12 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:12.624477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.630601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.640056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7d89a-1486-40aa-95fa-46eb2c98496e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.650991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:12.720374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.721002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.721284 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.721560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.721766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431e91-149e-4802-a854-dcbec6300e7b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.722241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.723918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.727928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.728564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.728839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.729107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.729806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.731754 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.738992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1277/2583] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:12.744039 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.756394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9f8fc-5459-4747-825d-49a620ca7fb9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.762073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.762761 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.763050 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.763303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] PATH_INFO: `/v3/projects/e319b10ccbc34925af87e9981cac1773/users/1415a98d95f14449ad3f32b37feb67b2/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.766827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1307/2584] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:12 2022] POST /identity/v3/users => generated 283 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:12.771030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.782643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9115b6e-753a-4af3-98ac-a8630c83e943 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.786931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e319b10ccbc34925af87e9981cac1773, user_id=1415a98d95f14449ad3f32b37feb67b2, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.789879 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.792009 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.792646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.792955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.793209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.793876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.795699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.811425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1278/2585] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:12 2022] PUT /identity/v3/projects/e319b10ccbc34925af87e9981cac1773/users/1415a98d95f14449ad3f32b37feb67b2/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:12.814178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1308/2586] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:12.819370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.819595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.831472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b081644-38f8-4b56-8f59-b8c663027cc1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.906107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc1bfd-3150-4d49-8b46-57cc42f0a751 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.911213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.911845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.912124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.912395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.913149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.915549 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.927696 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.928354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.928858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.929031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.929839 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.931559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.942734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1279/2587] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:12.947666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1309/2588] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:12.949469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.953020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:12.961905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df698d-7875-41fc-b06e-2c6bce2a33b5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.967026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef888d-4a18-49f1-aa8b-711edddfbdac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:12.970842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e851fb-322e-4a01-bfa1-166946ffa74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.971705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e851fb-322e-4a01-bfa1-166946ffa74a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.972022 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e851fb-322e-4a01-bfa1-166946ffa74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.972194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e851fb-322e-4a01-bfa1-166946ffa74a admin admin] PATH_INFO: `/v3/projects/e319b10ccbc34925af87e9981cac1773/users/1415a98d95f14449ad3f32b37feb67b2/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.973844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658f3e5-0f9c-4b5e-b143-324372e60548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:12.974340 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e851fb-322e-4a01-bfa1-166946ffa74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e319b10ccbc34925af87e9981cac1773, user_id=1415a98d95f14449ad3f32b37feb67b2, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:12.974814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658f3e5-0f9c-4b5e-b143-324372e60548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:12.975509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658f3e5-0f9c-4b5e-b143-324372e60548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:12.975927 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658f3e5-0f9c-4b5e-b143-324372e60548 admin admin] PATH_INFO: `/v3/projects/2d74ccb01ef44860a57accc4a48d44b1/users/cb9dd63d0bd94402a6af48eff618c257/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:12.977107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e851fb-322e-4a01-bfa1-166946ffa74a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:12.994461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1280/2589] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:12 2022] PUT /identity/v3/projects/e319b10ccbc34925af87e9981cac1773/users/1415a98d95f14449ad3f32b37feb67b2/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:13.003593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658f3e5-0f9c-4b5e-b143-324372e60548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d74ccb01ef44860a57accc4a48d44b1, user_id=cb9dd63d0bd94402a6af48eff618c257, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.005091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.005375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.005654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.006848 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658f3e5-0f9c-4b5e-b143-324372e60548 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.030707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1310/2590] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:12 2022] PUT /identity/v3/projects/2d74ccb01ef44860a57accc4a48d44b1/users/cb9dd63d0bd94402a6af48eff618c257/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:13.042876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca4d2-368a-49f3-8be0-1e4b9da4881d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.043196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca4d2-368a-49f3-8be0-1e4b9da4881d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.043402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca4d2-368a-49f3-8be0-1e4b9da4881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.046818 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] MFA Rules not processed for user `1415a98d95f14449ad3f32b37feb67b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:13.049590 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:13.075381 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:13.077706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d90ca4d2-368a-49f3-8be0-1e4b9da4881d None None] MFA Rules not processed for user `cb9dd63d0bd94402a6af48eff618c257`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:13.079750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d90ca4d2-368a-49f3-8be0-1e4b9da4881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:13.139782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:13.166223 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1281/2591] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:13.178851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:13.179387 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1311/2592] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:13.180980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:13.185524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ca4d2-368a-49f3-8be0-1e4b9da4881d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:13.254868 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728f3a-f927-41a0-8e8d-b50e0aa91ff1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:13.258471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ca4d2-368a-49f3-8be0-1e4b9da4881d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:13.259520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:13.260153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.260530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.260808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.265684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:13.266539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.267006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.267459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.268538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.271821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.275175 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.276920 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.287167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1282/2593] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:13.313781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1312/2594] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:13 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:13.319764 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:13.334660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2242-79e1-4f77-a72a-a21bbc2beb38 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:13.447206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:13.448732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.448870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.449978 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.451206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.454279 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.496722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1283/2595] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:13 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:13.501947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:13.516803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c34d8-f965-4b68-93a8-5bd6667be7d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:13.523738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:13.523738 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.524195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.524477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.525162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.532078 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.547643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1313/2596] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:13.549242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:13.566709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe45a-36fa-46da-8545-07fc09550a38 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:13.572899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:13.573771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.573979 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.574330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] PATH_INFO: `/v3/projects/13ca1c07e3f549f99d5a60694b666798/users/0ba1cbd3d3e941f8a0c5bdfbe6b8488d/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.595719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ca1c07e3f549f99d5a60694b666798, user_id=0ba1cbd3d3e941f8a0c5bdfbe6b8488d, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.601265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.625985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1284/2597] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:13 2022] PUT /identity/v3/projects/13ca1c07e3f549f99d5a60694b666798/users/0ba1cbd3d3e941f8a0c5bdfbe6b8488d/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:13.628607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:13.643645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81672d77-13b4-4a02-ab3b-69931a65b322 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:13.728870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee11a1-5a7e-4f95-8068-da3d1470c8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:13.732984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee11a1-5a7e-4f95-8068-da3d1470c8ea admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.732984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee11a1-5a7e-4f95-8068-da3d1470c8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.732984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee11a1-5a7e-4f95-8068-da3d1470c8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.732984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee11a1-5a7e-4f95-8068-da3d1470c8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.732984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee11a1-5a7e-4f95-8068-da3d1470c8ea admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.747780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1314/2598] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:13.752261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:13.763377 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382289-5c24-4d64-bf17-106846e99c13 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:13.768086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:13.768748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.769049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.769330 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] PATH_INFO: `/v3/projects/13ca1c07e3f549f99d5a60694b666798/users/0ba1cbd3d3e941f8a0c5bdfbe6b8488d/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.772094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ca1c07e3f549f99d5a60694b666798, user_id=0ba1cbd3d3e941f8a0c5bdfbe6b8488d, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:13.775111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:13.787384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1285/2599] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:13 2022] PUT /identity/v3/projects/13ca1c07e3f549f99d5a60694b666798/users/0ba1cbd3d3e941f8a0c5bdfbe6b8488d/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:13.800206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:13.800206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:13.800206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:13.831817 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:13.835349 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] MFA Rules not processed for user `0ba1cbd3d3e941f8a0c5bdfbe6b8488d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:13.837454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:13.929742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1315/2600] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:13.941992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:14.006535 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:14.013203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc95df-a45b-4ef7-9d3d-afc87ac05a7c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:14.017369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:14.018253 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:14.018628 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:14.018965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:14.031378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:14.033201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:14.043793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1286/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:14.126329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:14.128017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:14.142527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c861d85-267a-470f-b34a-c4fc4b17a2e9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:14.146712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303dec0-bbaa-48e9-b6f1-b5a0986745f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:14.147829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303dec0-bbaa-48e9-b6f1-b5a0986745f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:14.148171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303dec0-bbaa-48e9-b6f1-b5a0986745f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:14.148511 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303dec0-bbaa-48e9-b6f1-b5a0986745f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:14.216528 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303dec0-bbaa-48e9-b6f1-b5a0986745f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:14.218037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303dec0-bbaa-48e9-b6f1-b5a0986745f9 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:14.229096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1316/2602] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:14.961487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:15.037576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0dbba-1d6d-4706-96c5-b59d29cc05d6 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:15.043108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:15.044193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:15.044193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:15.044522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] PATH_INFO: `/v3/users/2ca42b910fc84bde9e82db1905fab6b4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:15.045862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca42b910fc84bde9e82db1905fab6b4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:15.048574 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:15.122216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca42b910fc84bde9e82db1905fab6b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.134421 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca42b910fc84bde9e82db1905fab6b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.152124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca42b910fc84bde9e82db1905fab6b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.153102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca42b910fc84bde9e82db1905fab6b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.168578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca42b910fc84bde9e82db1905fab6b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.179680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1287/2603] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:14 2022] DELETE /identity/v3/users/2ca42b910fc84bde9e82db1905fab6b4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:15.185610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:15.186193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:15.186193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:15.209790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:15.211597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:15.289740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1317/2604] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:15.302081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:15.311449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:15.385266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d2cf3-8250-4127-9c68-7e6c42ac12c7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:15.389086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:15.389889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:15.390013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:15.390270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:15.400837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:15.402303 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:15.415081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1288/2605] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:15.744877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:15.746873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:15.759935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fe95d-1b62-4678-8a02-43d563eb2b31 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:15.766129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:15.767011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:15.767441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:15.767857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] PATH_INFO: `/v3/projects/718cdda592474987ab7afb3eb0576e0f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:15.769228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=718cdda592474987ab7afb3eb0576e0f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:15.771703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:15.815819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:15.836360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718cdda592474987ab7afb3eb0576e0f', 'request_id': 'req-aed4e0bd-7bf2-4059-8326-4c0292403c4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.843761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718cdda592474987ab7afb3eb0576e0f', 'request_id': 'req-aed4e0bd-7bf2-4059-8326-4c0292403c4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.852725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718cdda592474987ab7afb3eb0576e0f', 'request_id': 'req-aed4e0bd-7bf2-4059-8326-4c0292403c4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.867298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] The token cache is being invalidate because project 718cdda592474987ab7afb3eb0576e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:15.867780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.868446 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:15.870418 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1318/2606] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:15 2022] DELETE /identity/v3/projects/718cdda592474987ab7afb3eb0576e0f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:15.874385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:15.884522 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1119d4-0d15-4b28-b46a-b319e043ec3f service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:15.892760 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:15.956951 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:15.957517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:15.957839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:15.958140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] PATH_INFO: `/v3/users/1415a98d95f14449ad3f32b37feb67b2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:15.959415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=1415a98d95f14449ad3f32b37feb67b2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:15.962130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:16.020214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1415a98d95f14449ad3f32b37feb67b2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.032854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1415a98d95f14449ad3f32b37feb67b2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.054377 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1415a98d95f14449ad3f32b37feb67b2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.055934 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1415a98d95f14449ad3f32b37feb67b2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.065644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b2ad9535-9b81-44e3-af35-300cafe5c0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1415a98d95f14449ad3f32b37feb67b2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.072701 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1289/2607] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:15 2022] DELETE /identity/v3/users/1415a98d95f14449ad3f32b37feb67b2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:16.410222 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:16.412453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:16.433693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4e0bd-7bf2-4059-8326-4c0292403c4a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:16.523137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:16.523614 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:16.523896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:16.524171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] PATH_INFO: `/v3/projects/e319b10ccbc34925af87e9981cac1773` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:16.525395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e319b10ccbc34925af87e9981cac1773)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:16.527541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:16.605806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e319b10ccbc34925af87e9981cac1773', 'request_id': 'req-f46fab39-cecc-4540-a972-447726eaf13e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.617423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e319b10ccbc34925af87e9981cac1773', 'request_id': 'req-f46fab39-cecc-4540-a972-447726eaf13e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.628056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e319b10ccbc34925af87e9981cac1773', 'request_id': 'req-f46fab39-cecc-4540-a972-447726eaf13e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.642067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] The token cache is being invalidate because project e319b10ccbc34925af87e9981cac1773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:16.642460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.643158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f46fab39-cecc-4540-a972-447726eaf13e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.644735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1319/2608] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:16 2022] DELETE /identity/v3/projects/e319b10ccbc34925af87e9981cac1773 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:16.651454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38b582-4622-4712-90ae-3589dbf937ce None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:16.651867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38b582-4622-4712-90ae-3589dbf937ce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:16.652257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38b582-4622-4712-90ae-3589dbf937ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:16.675371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8c38b582-4622-4712-90ae-3589dbf937ce None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:16.676685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8c38b582-4622-4712-90ae-3589dbf937ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:16.751826 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1290/2609] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:16.762240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362fd66-7c18-440e-934b-185046e2956c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:16.762883 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362fd66-7c18-440e-934b-185046e2956c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:16.764045 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362fd66-7c18-440e-934b-185046e2956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:16.787226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9362fd66-7c18-440e-934b-185046e2956c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:16.788558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9362fd66-7c18-440e-934b-185046e2956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:16.797215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1320/2610] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:16.802185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38b582-4622-4712-90ae-3589dbf937ce None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:16.815168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38b582-4622-4712-90ae-3589dbf937ce None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:16.821624 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:16.822449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:16.822819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:16.823143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:16.823841 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:16.824423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:16.826106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:16.845766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1291/2611] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:16.851410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362fd66-7c18-440e-934b-185046e2956c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:16.862026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362fd66-7c18-440e-934b-185046e2956c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:16.867736 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:16.868404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:16.868682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:16.869053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:16.869835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:16.872054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:16.908097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1321/2612] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:16 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:16.913048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:16.925871 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76010812-ba08-45ea-a6f0-e0c30a8bbdef admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:16.941520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:17.010626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.011385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.011788 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.012165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.012948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.015276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.048469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1292/2613] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:16 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:17.049733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.062297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabca7f7-e14b-4852-95b7-0be8a2d00888 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.067096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.068098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.068098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.068519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.069601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.071319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.086803 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1322/2614] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:17.089132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.111927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502c47d-5b2a-4284-a580-15719df291fe admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.115758 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.116991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.117485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.117983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.119147 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.122854 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.127460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.129287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:17.139988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1293/2615] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:17.143679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e35ee9-789f-429c-8191-ef7e7c3e362c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.144716 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.150309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.150934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.151216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.151496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] PATH_INFO: `/v3/users/cb9dd63d0bd94402a6af48eff618c257` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.152744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9dd63d0bd94402a6af48eff618c257)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.155139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.160926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:17.162604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ac37c-8fa2-404b-9ef9-0aaf749f5b57 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.170604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891104c-e597-46de-b9dc-755fdb0a100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.171707 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891104c-e597-46de-b9dc-755fdb0a100e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.172134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891104c-e597-46de-b9dc-755fdb0a100e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.172602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891104c-e597-46de-b9dc-755fdb0a100e admin admin] PATH_INFO: `/v3/projects/0f872149e1544145913c1423a20d510e/users/437ce362492848549b609295eb0301b9/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.201692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891104c-e597-46de-b9dc-755fdb0a100e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f872149e1544145913c1423a20d510e, user_id=437ce362492848549b609295eb0301b9, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.204648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891104c-e597-46de-b9dc-755fdb0a100e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.221119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9dd63d0bd94402a6af48eff618c257'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.222905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1294/2616] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:17 2022] PUT /identity/v3/projects/0f872149e1544145913c1423a20d510e/users/437ce362492848549b609295eb0301b9/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:17.239699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9dd63d0bd94402a6af48eff618c257'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.241266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.241926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.242754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.254661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9dd63d0bd94402a6af48eff618c257'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.255776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9dd63d0bd94402a6af48eff618c257'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.268077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-78e86262-9279-498f-a328-a5c8d9353301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9dd63d0bd94402a6af48eff618c257'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.277146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1323/2617] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:17 2022] DELETE /identity/v3/users/cb9dd63d0bd94402a6af48eff618c257 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:17.282742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455745ec-1597-47b1-a216-40823394c112 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.283649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455745ec-1597-47b1-a216-40823394c112 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.284134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455745ec-1597-47b1-a216-40823394c112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.285657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] MFA Rules not processed for user `437ce362492848549b609295eb0301b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:17.288121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:17.316333 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-455745ec-1597-47b1-a216-40823394c112 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:17.318090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-455745ec-1597-47b1-a216-40823394c112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:17.388427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1295/2618] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:17.393141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.394661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:17.419302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1324/2619] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:17.432485 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455745ec-1597-47b1-a216-40823394c112 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.435204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-455745ec-1597-47b1-a216-40823394c112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:17.482059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1f732-7b2d-427a-a24a-2365130d50b1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.490148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.491077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.491473 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.491704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.492302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.494308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.500865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:17.517130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455745ec-1597-47b1-a216-40823394c112 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.525695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.526566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.529807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.529807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.545413 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.547568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1296/2620] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:17 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:17.549024 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.554986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.567425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1325/2621] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:17.673128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eccdd-792a-478f-ae10-a93b5960dec9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.681539 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.681885 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.682227 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.682544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.683255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.685836 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.717913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1297/2622] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:17 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:17.722147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.735801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f01a5-1301-44ca-a392-bf5bedb92788 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.758237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.759211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.759725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.761019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.774084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.783217 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.800828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1326/2623] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:17.811965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.829500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343ddb-30cb-4b4a-be6d-1def5a313a54 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.840635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.842523 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.842950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.843415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.844642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.848067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.868653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1298/2624] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:17.871152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.906665 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06920f76-1546-43ec-bc23-6c8a31f541fc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.913392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cb631-1e41-4a7c-90e5-00733e9380a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.914507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cb631-1e41-4a7c-90e5-00733e9380a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.914989 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cb631-1e41-4a7c-90e5-00733e9380a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.915702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cb631-1e41-4a7c-90e5-00733e9380a6 admin admin] PATH_INFO: `/v3/projects/ffd072ac454646fca4e61861dbed05d5/users/09d1f2281ef54b5888db0f4c05f04939/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.938259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:17.957841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cb631-1e41-4a7c-90e5-00733e9380a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd072ac454646fca4e61861dbed05d5, user_id=09d1f2281ef54b5888db0f4c05f04939, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.968561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cb631-1e41-4a7c-90e5-00733e9380a6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.035111 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1327/2625] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:17 2022] PUT /identity/v3/projects/ffd072ac454646fca4e61861dbed05d5/users/09d1f2281ef54b5888db0f4c05f04939/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:18.047179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.048108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:18.051460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbcab0-5247-49e9-a4de-79c727eedec3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.057189 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbcab0-5247-49e9-a4de-79c727eedec3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.057416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbcab0-5247-49e9-a4de-79c727eedec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.082996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeafac-00ba-436a-b922-eedaca97457c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.126545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-93bbcab0-5247-49e9-a4de-79c727eedec3 None None] MFA Rules not processed for user `09d1f2281ef54b5888db0f4c05f04939`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:18.128644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-93bbcab0-5247-49e9-a4de-79c727eedec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:18.203112 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.204573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.204573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.205696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] PATH_INFO: `/v3/projects/2d74ccb01ef44860a57accc4a48d44b1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.206734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d74ccb01ef44860a57accc4a48d44b1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.209620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.247202 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1328/2626] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:18.254849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbcab0-5247-49e9-a4de-79c727eedec3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.268880 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbcab0-5247-49e9-a4de-79c727eedec3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.274622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.275248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.275530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.275810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.276483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.278479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.291670 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d74ccb01ef44860a57accc4a48d44b1', 'request_id': 'req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.306449 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d74ccb01ef44860a57accc4a48d44b1', 'request_id': 'req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.320286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1329/2627] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:18 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:18.320559 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d74ccb01ef44860a57accc4a48d44b1', 'request_id': 'req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.322282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.324674 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:18.339636 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] The token cache is being invalidate because project 2d74ccb01ef44860a57accc4a48d44b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:18.340116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.341588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.343925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1299/2628] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:18 2022] DELETE /identity/v3/projects/2d74ccb01ef44860a57accc4a48d44b1 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:18.349045 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.351377 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:18.436135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b202c-176c-4d21-9d71-94fb2eaafca7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.444688 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.445759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.446233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.446652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.447572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.451056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.488811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1330/2629] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:18 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:18.495587 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.498592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:18.505618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244cd8b-45aa-48a4-9320-a4a5a9fbe33e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.514544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.515673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.516177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.516662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] PATH_INFO: `/v3/users/0ba1cbd3d3e941f8a0c5bdfbe6b8488d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.518784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ba1cbd3d3e941f8a0c5bdfbe6b8488d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.521382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5690-2aca-42ae-9e54-c34502db77bf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.522825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.529198 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.530079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.530565 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.530999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.532073 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.534886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.553159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1331/2630] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:18.560133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.578660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8eb2-32a9-4ef4-9368-3739d116c93a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.587795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.589632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.590490 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.591416 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] PATH_INFO: `/v3/projects/aabae08227a44e2ea99e8603e96de99a/users/9458b797aba24c068e137a1e34520d3b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.623407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabae08227a44e2ea99e8603e96de99a, user_id=9458b797aba24c068e137a1e34520d3b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.629018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.633080 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ba1cbd3d3e941f8a0c5bdfbe6b8488d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.645439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ba1cbd3d3e941f8a0c5bdfbe6b8488d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.653335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1332/2631] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:18 2022] PUT /identity/v3/projects/aabae08227a44e2ea99e8603e96de99a/users/9458b797aba24c068e137a1e34520d3b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:18.661040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.675511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ba1cbd3d3e941f8a0c5bdfbe6b8488d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.676992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ba1cbd3d3e941f8a0c5bdfbe6b8488d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.683788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:18.693796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ba1cbd3d3e941f8a0c5bdfbe6b8488d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.702028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1300/2632] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:18 2022] DELETE /identity/v3/users/0ba1cbd3d3e941f8a0c5bdfbe6b8488d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:18.914889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9c8ab-cd2e-4124-b039-9ceb97f3f668 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.004821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1c81c-2752-4e4c-b81d-551b95515a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.005531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c81c-2752-4e4c-b81d-551b95515a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.005960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c81c-2752-4e4c-b81d-551b95515a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.006344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c81c-2752-4e4c-b81d-551b95515a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.007065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1c81c-2752-4e4c-b81d-551b95515a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.008789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1c81c-2752-4e4c-b81d-551b95515a9e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.023844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1333/2633] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:19.027825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.029405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.046814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4031cc-b0cc-4bda-b151-d79ba38883ff admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.052095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.053102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.053102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.053393 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] PATH_INFO: `/v3/projects/aabae08227a44e2ea99e8603e96de99a/users/9458b797aba24c068e137a1e34520d3b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.055933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabae08227a44e2ea99e8603e96de99a, user_id=9458b797aba24c068e137a1e34520d3b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.058201 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.073461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1301/2634] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:19 2022] PUT /identity/v3/projects/aabae08227a44e2ea99e8603e96de99a/users/9458b797aba24c068e137a1e34520d3b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:19.085602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.086065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.086497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.119753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.125857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.154604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] MFA Rules not processed for user `9458b797aba24c068e137a1e34520d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:19.157271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.300837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57285e1b-6717-417a-add9-fabe29623aac admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.309078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.310199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.310673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.311146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] PATH_INFO: `/v3/projects/13ca1c07e3f549f99d5a60694b666798` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.313159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ca1c07e3f549f99d5a60694b666798)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.316747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.324741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:19.345597 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1334/2635] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:19.367953 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.370169 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.429570 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ca1c07e3f549f99d5a60694b666798', 'request_id': 'req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:19.437927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ca1c07e3f549f99d5a60694b666798', 'request_id': 'req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:19.447774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ca1c07e3f549f99d5a60694b666798', 'request_id': 'req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:19.460080 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:19.479071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] The token cache is being invalidate because project 13ca1c07e3f549f99d5a60694b666798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:19.479591 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:19.480454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ccf19870-29dc-4f6d-80ba-f8cc72c7ef56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:19.482266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1302/2636] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:19 2022] DELETE /identity/v3/projects/13ca1c07e3f549f99d5a60694b666798 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:19.487220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef396f2c-a844-4c3f-a304-7766f300b66d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.490143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacb15-21ee-4e01-8e07-10114b3309ac None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.490143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacb15-21ee-4e01-8e07-10114b3309ac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.490558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacb15-21ee-4e01-8e07-10114b3309ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.522899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f0eacb15-21ee-4e01-8e07-10114b3309ac None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:19.524884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f0eacb15-21ee-4e01-8e07-10114b3309ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.574854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.575550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.575823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.576088 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.579773 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.634471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1303/2637] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:19.643111 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f210b7-6b2d-49d4-ba3b-6642d0210e29 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.643424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f210b7-6b2d-49d4-ba3b-6642d0210e29 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.643679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f210b7-6b2d-49d4-ba3b-6642d0210e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.678774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f5f210b7-6b2d-49d4-ba3b-6642d0210e29 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:19.681931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f5f210b7-6b2d-49d4-ba3b-6642d0210e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.695555 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.697287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.701248 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1304/2638] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:19.702960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f210b7-6b2d-49d4-ba3b-6642d0210e29 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.712870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1335/2639] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:19.716563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f210b7-6b2d-49d4-ba3b-6642d0210e29 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.725784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.728160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.728160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.728160 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.728637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:19.729255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.732047 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.754921 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1305/2640] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:19.760608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.775151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fad9a-bab6-4f67-84ab-15def9e5235a service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.783344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.783972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.784346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.784557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.785257 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.787611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.830742 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1336/2641] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:19.835445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.848848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c297bb-c6a9-4cb3-8b45-7f2820e1bee5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.954616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.955304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.955517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.955823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.956745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.959099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.988748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1306/2642] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:19.995130 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.007316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4537a-9c9c-4ef1-af52-436d3cab0692 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.021600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.022187 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.022533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.022982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.023979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.026228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.048843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1337/2643] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:20.050466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.057409 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:20.069647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d363e-1721-4e55-b10d-e2956a421a55 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.079289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ff735-4335-4aca-b885-6806acca6997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.080784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff735-4335-4aca-b885-6806acca6997 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.081811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff735-4335-4aca-b885-6806acca6997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.081811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff735-4335-4aca-b885-6806acca6997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.082102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ff735-4335-4aca-b885-6806acca6997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.084622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ff735-4335-4aca-b885-6806acca6997 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.100724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1307/2644] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:20.105071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.122285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0b215-fc12-4c4c-9e4f-76f293b6c5e0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.129637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.134905 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.135052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.135265 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] PATH_INFO: `/v3/projects/1318bbbdfa994145aecc4d00b1a12eff/users/02b71cc776014e47bcb4c666f2c44f5e/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.153452 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1318bbbdfa994145aecc4d00b1a12eff, user_id=02b71cc776014e47bcb4c666f2c44f5e, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.156713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.192269 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1338/2645] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:20 2022] PUT /identity/v3/projects/1318bbbdfa994145aecc4d00b1a12eff/users/02b71cc776014e47bcb4c666f2c44f5e/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:20.208622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a237-a485-4f8c-9a91-eb0b4b8da932 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.208622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a237-a485-4f8c-9a91-eb0b4b8da932 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.208622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880a237-a485-4f8c-9a91-eb0b4b8da932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.241725 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7880a237-a485-4f8c-9a91-eb0b4b8da932 None None] MFA Rules not processed for user `02b71cc776014e47bcb4c666f2c44f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:20.244048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7880a237-a485-4f8c-9a91-eb0b4b8da932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:20.343819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1308/2646] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:20.349133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.434897 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:20.438890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50408af3-0bab-4901-862e-adc51473e4b9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.446352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.447001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.447275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.447541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.448288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.450268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.492681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1339/2647] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:20 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:20.497059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880a237-a485-4f8c-9a91-eb0b4b8da932 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.509656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880a237-a485-4f8c-9a91-eb0b4b8da932 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.596033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.596665 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.597004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.597490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.598170 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.600645 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.631924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1309/2648] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:20 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:20.633273 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.647324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2b4e3-0b50-4465-9bd6-62c12b72aef2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.653487 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.654151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.654433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.654705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.655353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.657067 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.670231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1340/2649] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:20.675202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.687049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580d8cd-7f3e-4d17-ae6e-cd4a72ac3cd2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.692083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.692877 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.693362 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.693808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] PATH_INFO: `/v3/projects/a1636126f15e40fe85a33bb4985863e3/users/d4e9e8039f97445296dc9be9919f5588/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.726588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1636126f15e40fe85a33bb4985863e3, user_id=d4e9e8039f97445296dc9be9919f5588, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.730639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.747516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1310/2650] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:20 2022] PUT /identity/v3/projects/a1636126f15e40fe85a33bb4985863e3/users/d4e9e8039f97445296dc9be9919f5588/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:20.752392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.767258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3e111-1e60-47c7-bbae-0cf13d752008 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.848031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fc1e7-9245-4634-bab6-ea5f5666a63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.848658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc1e7-9245-4634-bab6-ea5f5666a63f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.849086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc1e7-9245-4634-bab6-ea5f5666a63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.849621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc1e7-9245-4634-bab6-ea5f5666a63f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.850318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fc1e7-9245-4634-bab6-ea5f5666a63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.851950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fc1e7-9245-4634-bab6-ea5f5666a63f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.864920 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1341/2651] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:20.869140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:20.882478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2251d9d-6735-4dcd-8593-3c368a52d97d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:20.888631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:20.889609 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.889983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.890328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] PATH_INFO: `/v3/projects/a1636126f15e40fe85a33bb4985863e3/users/d4e9e8039f97445296dc9be9919f5588/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.894677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1636126f15e40fe85a33bb4985863e3, user_id=d4e9e8039f97445296dc9be9919f5588, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:20.898225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:20.913766 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1311/2652] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:20 2022] PUT /identity/v3/projects/a1636126f15e40fe85a33bb4985863e3/users/d4e9e8039f97445296dc9be9919f5588/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:20.921175 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d450f39-e94c-4050-a043-abc2422e46ea None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:20.921514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d450f39-e94c-4050-a043-abc2422e46ea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:20.921964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d450f39-e94c-4050-a043-abc2422e46ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:20.959467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4d450f39-e94c-4050-a043-abc2422e46ea None None] MFA Rules not processed for user `d4e9e8039f97445296dc9be9919f5588`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:20.960941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4d450f39-e94c-4050-a043-abc2422e46ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:21.056327 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1342/2653] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:21.071392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:21.073257 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:21.112173 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:21.149068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0e6f4-d07e-4382-9814-488127bb0769 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:21.153740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:21.154354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:21.154659 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:21.154929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:21.166058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:21.167511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:21.177973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1312/2654] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:21.233833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d450f39-e94c-4050-a043-abc2422e46ea None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:21.245315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d450f39-e94c-4050-a043-abc2422e46ea None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:21.250325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:21.251223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:21.251572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:21.251915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:21.312243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:21.324442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:21.326199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:21.335819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1343/2655] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:21.438677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:21.440136 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:21.452128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44437a-a66f-494b-8c39-ef5acfc65560 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:21.455832 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:21.456439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:21.456721 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:21.456995 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:21.458785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:21.527085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:21.528639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:21.540090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1313/2656] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:22.182438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:22.192131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06448da-01a5-499f-b0fd-8f9370694420 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:22.196126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:22.196771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:22.197046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:22.197337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:22.202651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:22.281639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:22.283134 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:22.293133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1344/2657] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:24.902697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.983371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aaf80-0e2e-4a3a-a316-3c1b03802097 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:24.995795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.998029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:24.998986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:24.999745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:25.000121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:25.000484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] PATH_INFO: `/v3/users/02b71cc776014e47bcb4c666f2c44f5e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:25.001779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b71cc776014e47bcb4c666f2c44f5e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:25.004572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:25.017087 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4563e9-348c-41a5-8422-2dbe3bc0d125 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:25.025165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:25.027004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:25.027407 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:25.027843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] PATH_INFO: `/v3/users/83d4fc784bca4f618f6b16a226a22aa7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:25.029177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d4fc784bca4f618f6b16a226a22aa7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:25.031164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:25.070127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b71cc776014e47bcb4c666f2c44f5e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.078460 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b71cc776014e47bcb4c666f2c44f5e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.089297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d4fc784bca4f618f6b16a226a22aa7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.098479 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b71cc776014e47bcb4c666f2c44f5e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.099679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b71cc776014e47bcb4c666f2c44f5e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.105346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:25.106364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d4fc784bca4f618f6b16a226a22aa7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.109274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-341c9e0a-29d8-48ac-8a75-666993d807a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b71cc776014e47bcb4c666f2c44f5e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.115922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d4fc784bca4f618f6b16a226a22aa7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.116912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d4fc784bca4f618f6b16a226a22aa7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.118837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1314/2658] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:24 2022] DELETE /identity/v3/users/02b71cc776014e47bcb4c666f2c44f5e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:25.128033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:25.128033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:25.128033 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:25.135984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d4fc784bca4f618f6b16a226a22aa7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.144734 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1345/2659] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:24 2022] DELETE /identity/v3/users/83d4fc784bca4f618f6b16a226a22aa7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:25.146178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:25.147810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:25.223884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1315/2660] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:25.233972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:25.235488 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:25.303592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa8568-1c3a-4905-9911-6b0118212e3c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:25.307216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:25.307829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:25.308104 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:25.308373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:25.321209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:25.322858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:25.327477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:25.336243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1346/2661] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:25.337290 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:25.338952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:25.352299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251523d-90fb-4bb4-9566-95263f25579b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:25.359454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:25.360441 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:25.360816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:25.361187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] PATH_INFO: `/v3/projects/f47ce668d54145b8a589391b4e0092a7` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:25.362658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47ce668d54145b8a589391b4e0092a7)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:25.364966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:25.554145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47ce668d54145b8a589391b4e0092a7', 'request_id': 'req-3fa1721c-08ae-4ac6-a429-80c88fec8663'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.564718 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47ce668d54145b8a589391b4e0092a7', 'request_id': 'req-3fa1721c-08ae-4ac6-a429-80c88fec8663'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.572365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47ce668d54145b8a589391b4e0092a7', 'request_id': 'req-3fa1721c-08ae-4ac6-a429-80c88fec8663'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.586275 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] The token cache is being invalidate because project f47ce668d54145b8a589391b4e0092a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:25.586536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.587482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.589699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1316/2662] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:25 2022] DELETE /identity/v3/projects/f47ce668d54145b8a589391b4e0092a7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:25.593771 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:25.604179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b3f6f-30b9-4618-833d-65945768bc2b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:25.653252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:25.654748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:25.677068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:25.677747 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:25.678535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:25.678535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] PATH_INFO: `/v3/users/0f24760ac5494571838ed1002efcda4e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:25.679466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f24760ac5494571838ed1002efcda4e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:25.681357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:25.736386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1721c-08ae-4ac6-a429-80c88fec8663 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:25.737558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f24760ac5494571838ed1002efcda4e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.744529 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:25.745518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:25.745988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:25.746413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] PATH_INFO: `/v3/projects/1318bbbdfa994145aecc4d00b1a12eff` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:25.748182 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1318bbbdfa994145aecc4d00b1a12eff)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:25.751506 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:25.756850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f24760ac5494571838ed1002efcda4e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.767648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f24760ac5494571838ed1002efcda4e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.768500 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f24760ac5494571838ed1002efcda4e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.776802 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f24760ac5494571838ed1002efcda4e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.780931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:25.784182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1347/2663] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:25 2022] DELETE /identity/v3/users/0f24760ac5494571838ed1002efcda4e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:25.820563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1318bbbdfa994145aecc4d00b1a12eff', 'request_id': 'req-78cc861f-ab52-4b2b-a536-44958d10fc06'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.828511 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1318bbbdfa994145aecc4d00b1a12eff', 'request_id': 'req-78cc861f-ab52-4b2b-a536-44958d10fc06'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.837552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1318bbbdfa994145aecc4d00b1a12eff', 'request_id': 'req-78cc861f-ab52-4b2b-a536-44958d10fc06'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.857071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] The token cache is being invalidate because project 1318bbbdfa994145aecc4d00b1a12eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:25.857520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.858347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:25.860152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1317/2664] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:25 2022] DELETE /identity/v3/projects/1318bbbdfa994145aecc4d00b1a12eff => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:25.865403 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:25.867128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:25.941722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea96745-622e-4e02-9367-62ce1dcbe790 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:25.970893 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:25.987475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:25.988967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.022123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.022763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.023040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.023311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] PATH_INFO: `/v3/users/d4e9e8039f97445296dc9be9919f5588` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.024459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e9e8039f97445296dc9be9919f5588)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.026676 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.065113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc861f-ab52-4b2b-a536-44958d10fc06 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.070166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.070785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.071150 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.071424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] PATH_INFO: `/v3/projects/278e22734c57426fa8c760ebddb039a5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.072732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278e22734c57426fa8c760ebddb039a5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.074682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.082285 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e9e8039f97445296dc9be9919f5588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.098090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e9e8039f97445296dc9be9919f5588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.109337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e9e8039f97445296dc9be9919f5588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.110095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e9e8039f97445296dc9be9919f5588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.118405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2add6f1-2c12-424d-ba23-1a5cda2a0670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e9e8039f97445296dc9be9919f5588'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.125145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1348/2665] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:25 2022] DELETE /identity/v3/users/d4e9e8039f97445296dc9be9919f5588 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:26.133348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278e22734c57426fa8c760ebddb039a5', 'request_id': 'req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.139471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278e22734c57426fa8c760ebddb039a5', 'request_id': 'req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.146736 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278e22734c57426fa8c760ebddb039a5', 'request_id': 'req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.164332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] The token cache is being invalidate because project 278e22734c57426fa8c760ebddb039a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:26.164685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.165268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-91aa1724-ca41-41f7-9871-cbaa07c5c3a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.167166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1318/2666] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:25 2022] DELETE /identity/v3/projects/278e22734c57426fa8c760ebddb039a5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:26.172882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29563aac-df58-4387-a084-09dd569fcc05 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.173190 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29563aac-df58-4387-a084-09dd569fcc05 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.174182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29563aac-df58-4387-a084-09dd569fcc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.197786 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-29563aac-df58-4387-a084-09dd569fcc05 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:26.199013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-29563aac-df58-4387-a084-09dd569fcc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.266728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1349/2667] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:26.276236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36954821-183d-4736-9be4-33518c3863a9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.276612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36954821-183d-4736-9be4-33518c3863a9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.277023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36954821-183d-4736-9be4-33518c3863a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.309031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-36954821-183d-4736-9be4-33518c3863a9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:26.311543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-36954821-183d-4736-9be4-33518c3863a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.332055 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1319/2668] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:26.333309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29563aac-df58-4387-a084-09dd569fcc05 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.347577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29563aac-df58-4387-a084-09dd569fcc05 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.352572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.353241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.353451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.353766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.354493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:26.354693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.356334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.372209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36954821-183d-4736-9be4-33518c3863a9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.373715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-36954821-183d-4736-9be4-33518c3863a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.374284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1350/2669] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:26.378419 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.392370 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36954821-183d-4736-9be4-33518c3863a9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.397562 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14016d3-ad1e-4351-9ffc-aa55190443d5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.404835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.405456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.405768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.406034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.406683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.408729 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.412316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.414137 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.415964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.416386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] PATH_INFO: `/v3/users/437ce362492848549b609295eb0301b9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.418158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=437ce362492848549b609295eb0301b9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.420235 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.454328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1351/2670] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:26.456268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:26.456612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.472223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:26.490790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437ce362492848549b609295eb0301b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.503536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437ce362492848549b609295eb0301b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.523486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437ce362492848549b609295eb0301b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.524271 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437ce362492848549b609295eb0301b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.533527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437ce362492848549b609295eb0301b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.540711 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1320/2671] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:26 2022] DELETE /identity/v3/users/437ce362492848549b609295eb0301b9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:26.541534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.543159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.548992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee3330-6c8e-47f8-bc42-6709e75dc3e9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.611609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf8d6a-5c2b-4dea-98ad-1d90fa0289a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.612113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf8d6a-5c2b-4dea-98ad-1d90fa0289a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.612395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf8d6a-5c2b-4dea-98ad-1d90fa0289a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.612667 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf8d6a-5c2b-4dea-98ad-1d90fa0289a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.613386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf8d6a-5c2b-4dea-98ad-1d90fa0289a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.615342 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf8d6a-5c2b-4dea-98ad-1d90fa0289a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.618475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed8e83-503c-4960-a086-33e2b23b9b94 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.622984 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.623690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.623962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.624252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] PATH_INFO: `/v3/projects/a1636126f15e40fe85a33bb4985863e3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.625339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1636126f15e40fe85a33bb4985863e3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.627282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.638133 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1352/2672] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:26.639466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.639763 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.640035 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.660237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:26.661561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.670443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1353/2673] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:26.673790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.673790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.679536 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1636126f15e40fe85a33bb4985863e3', 'request_id': 'req-267cfce0-71cd-4656-909f-ebd50efd6034'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.686254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1636126f15e40fe85a33bb4985863e3', 'request_id': 'req-267cfce0-71cd-4656-909f-ebd50efd6034'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.686496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7814-a896-4593-a28f-749f43ac05e9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.693304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1636126f15e40fe85a33bb4985863e3', 'request_id': 'req-267cfce0-71cd-4656-909f-ebd50efd6034'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.708132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] The token cache is being invalidate because project a1636126f15e40fe85a33bb4985863e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:26.708561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.709619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:26.711358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1321/2674] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:26 2022] DELETE /identity/v3/projects/a1636126f15e40fe85a33bb4985863e3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:26.712392 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.714689 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.750819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f11d4-85ad-4641-84f3-844db188838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.751442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f11d4-85ad-4641-84f3-844db188838c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.751719 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f11d4-85ad-4641-84f3-844db188838c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.751992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f11d4-85ad-4641-84f3-844db188838c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.752635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f11d4-85ad-4641-84f3-844db188838c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.753573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:26.754291 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f11d4-85ad-4641-84f3-844db188838c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.766137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1354/2675] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:26.767474 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.767759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.768030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.784752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cfce0-71cd-4656-909f-ebd50efd6034 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.788895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63782adb-96ac-47cd-87a3-71d0aa9d833a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.789135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:26.789515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63782adb-96ac-47cd-87a3-71d0aa9d833a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.789863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63782adb-96ac-47cd-87a3-71d0aa9d833a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.790113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63782adb-96ac-47cd-87a3-71d0aa9d833a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.790396 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.791918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-63782adb-96ac-47cd-87a3-71d0aa9d833a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.799531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1355/2676] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:26.806564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.806564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.806749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63782adb-96ac-47cd-87a3-71d0aa9d833a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.806749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63782adb-96ac-47cd-87a3-71d0aa9d833a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.817231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb75b0-96f6-4d69-8d58-83477eb689d5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.821003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1322/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:26.822528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bff931-2a18-464f-a67b-285407c949fb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.822813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bff931-2a18-464f-a67b-285407c949fb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.823089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bff931-2a18-464f-a67b-285407c949fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.825003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.825636 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.825962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.826230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.827096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.828865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.843816 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1356/2678] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:26.845200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.846720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e6bff931-2a18-464f-a67b-285407c949fb None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:26.847950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6bff931-2a18-464f-a67b-285407c949fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.858680 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66849091-e930-478f-a3da-e2153a49ef63 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.861881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1323/2679] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:26.863463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bff931-2a18-464f-a67b-285407c949fb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.867589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1411c10-073d-4fe4-b0b1-7df3c8f2e125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.867888 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411c10-073d-4fe4-b0b1-7df3c8f2e125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.868274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411c10-073d-4fe4-b0b1-7df3c8f2e125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.868640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411c10-073d-4fe4-b0b1-7df3c8f2e125 admin admin] PATH_INFO: `/v3/projects/11f1f3eefd684734900eb5e32a37bc85/users/14f98e16cd3244ceb66904aa082ef9f2/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.873385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bff931-2a18-464f-a67b-285407c949fb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.878637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.879572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.879572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.879861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.880514 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:26.880903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.882575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.888072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1411c10-073d-4fe4-b0b1-7df3c8f2e125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f1f3eefd684734900eb5e32a37bc85, user_id=14f98e16cd3244ceb66904aa082ef9f2, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.891339 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1411c10-073d-4fe4-b0b1-7df3c8f2e125 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:26.894259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:26.901709 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1324/2680] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:19:26.903001 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:26.920870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1357/2681] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:26 2022] PUT /identity/v3/projects/11f1f3eefd684734900eb5e32a37bc85/users/14f98e16cd3244ceb66904aa082ef9f2/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:26.921206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484685-7d28-496d-a822-aba45e05035d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:26.924635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.924944 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.925193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.959739 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] MFA Rules not processed for user `14f98e16cd3244ceb66904aa082ef9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:26.961064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:26.985504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:26.986323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:26.986661 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:26.986998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:26.987723 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:26.990083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.027141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1325/2682] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:27.028274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.047648 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1358/2683] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:27.059807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.061473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:27.105004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36952514-3212-4fc5-94b7-e5efd065c20e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.111387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.112193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.112484 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.112962 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.113690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.116310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.131071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a57f-5ec9-4606-b09d-1d24bb751472 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.144448 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1326/2684] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:27 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:27.148664 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.161846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef75f20-c410-41d8-9b1f-1873f41b20d8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.169762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.171167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.171167 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.171384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.172507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.176276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.189315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1327/2685] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:27.193941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.200998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.201795 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.202095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.202370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.206856 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b80848-28e6-4b46-a859-ffef000cdbcc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.216985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.217601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.218063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.218347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.219009 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.221839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.234063 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1328/2686] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:27.234964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.238347 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:27.252882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5649971-1d57-48b5-b7d0-bb6bd260139c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.258074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.259076 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.259489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.260084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] PATH_INFO: `/v3/projects/0f872149e1544145913c1423a20d510e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.261243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f872149e1544145913c1423a20d510e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.263199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.266572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:27.272577 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:27.277434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.280294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.291165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1359/2687] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:27.292096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.303451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff5a2f-e7e4-4753-8e9b-41a1bc50659d service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.309142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c298810-3779-4657-8583-0e3e03e30960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.309971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298810-3779-4657-8583-0e3e03e30960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.310372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298810-3779-4657-8583-0e3e03e30960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.310886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298810-3779-4657-8583-0e3e03e30960 admin admin] PATH_INFO: `/v3/projects/2da1efd0c041487488811ff5bc1f77e3/users/195b777b82194a0f879dc706afc9c89b/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.330784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c298810-3779-4657-8583-0e3e03e30960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da1efd0c041487488811ff5bc1f77e3, user_id=195b777b82194a0f879dc706afc9c89b, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.334252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c298810-3779-4657-8583-0e3e03e30960 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.337270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f872149e1544145913c1423a20d510e', 'request_id': 'req-c4d3d9c5-3293-450f-a308-457000db4007'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.344797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f872149e1544145913c1423a20d510e', 'request_id': 'req-c4d3d9c5-3293-450f-a308-457000db4007'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.352367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1360/2688] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:27 2022] PUT /identity/v3/projects/2da1efd0c041487488811ff5bc1f77e3/users/195b777b82194a0f879dc706afc9c89b/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:27.353806 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f872149e1544145913c1423a20d510e', 'request_id': 'req-c4d3d9c5-3293-450f-a308-457000db4007'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.364553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.365044 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.365410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.373964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] The token cache is being invalidate because project 0f872149e1544145913c1423a20d510e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:27.374280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.374860 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.382368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1329/2689] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:27 2022] DELETE /identity/v3/projects/0f872149e1544145913c1423a20d510e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:27.386812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.389696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:27.407712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] MFA Rules not processed for user `195b777b82194a0f879dc706afc9c89b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:27.408938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:27.474947 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d9c5-3293-450f-a308-457000db4007 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.493467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1361/2690] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:27.498025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.500127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:27.564974 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.566813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.567209 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.567617 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] PATH_INFO: `/v3/users/09d1f2281ef54b5888db0f4c05f04939` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.569179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d1f2281ef54b5888db0f4c05f04939)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.571174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.596165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eaa2d-ddb3-48a1-84f7-e5d92ffd9ab1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.603015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.604301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.604301 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.604922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.606379 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.609304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.634594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:27.643831 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1362/2691] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:27 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:27.646135 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.648592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d1f2281ef54b5888db0f4c05f04939'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.662577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d1f2281ef54b5888db0f4c05f04939'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.690419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d1f2281ef54b5888db0f4c05f04939'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.691267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d1f2281ef54b5888db0f4c05f04939'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.701531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d1f2281ef54b5888db0f4c05f04939'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:27.709278 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1330/2692] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:27 2022] DELETE /identity/v3/users/09d1f2281ef54b5888db0f4c05f04939 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:27.715679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:27.740043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030005f-8498-4bd3-8f3e-1be01921d3ff admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.837740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.838377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.838653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.838926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.839561 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.841653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.875679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1363/2693] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:27 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:27.882691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.884556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:27.900801 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813b4da-bec4-4862-8c58-756b172c5fee admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.911105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.912680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.913176 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.913496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.914311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.916255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.933143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1331/2694] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:27.937896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.953425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31518a8-fb63-49e0-acbb-f8717997eb7b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:27.960266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c7da8-4cf4-448f-9856-35aac219beea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:27.961246 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c7da8-4cf4-448f-9856-35aac219beea admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:27.962157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c7da8-4cf4-448f-9856-35aac219beea admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:27.962310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c7da8-4cf4-448f-9856-35aac219beea admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:27.963609 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c7da8-4cf4-448f-9856-35aac219beea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:27.966858 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c7da8-4cf4-448f-9856-35aac219beea admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:27.982531 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1364/2695] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:27.986896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:27.999747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703ca69-b897-4fe7-9343-cd71929315ea admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.008035 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.008926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.009179 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.009564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] PATH_INFO: `/v3/projects/3e618023d22348838e7427f816036de9/users/da65a2e4e1764868a8677ecaf05c3d85/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.030113 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e618023d22348838e7427f816036de9, user_id=da65a2e4e1764868a8677ecaf05c3d85, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.032888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.054853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1332/2696] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:27 2022] PUT /identity/v3/projects/3e618023d22348838e7427f816036de9/users/da65a2e4e1764868a8677ecaf05c3d85/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:28.062823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04ba17-c9a5-403a-9f0a-ac16e61ded8b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.063325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04ba17-c9a5-403a-9f0a-ac16e61ded8b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.063496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04ba17-c9a5-403a-9f0a-ac16e61ded8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.107807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5b04ba17-c9a5-403a-9f0a-ac16e61ded8b None None] MFA Rules not processed for user `da65a2e4e1764868a8677ecaf05c3d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:28.109280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5b04ba17-c9a5-403a-9f0a-ac16e61ded8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:28.116440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.118417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:28.201071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1365/2697] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:28.205656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04ba17-c9a5-403a-9f0a-ac16e61ded8b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.217889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04ba17-c9a5-403a-9f0a-ac16e61ded8b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.219006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04ee28-f0e3-44c3-9c6a-da616400a8a8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.222505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.223100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.223491 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.223672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.224475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.224605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.225123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.225248 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.225558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] PATH_INFO: `/v3/projects/ffd072ac454646fca4e61861dbed05d5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.227052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd072ac454646fca4e61861dbed05d5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.227197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.229291 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.263799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1366/2698] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:28 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:28.267764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.303469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd072ac454646fca4e61861dbed05d5', 'request_id': 'req-092ee285-b29d-4d9f-a03a-b2bb07294606'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.318116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd072ac454646fca4e61861dbed05d5', 'request_id': 'req-092ee285-b29d-4d9f-a03a-b2bb07294606'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.330620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd072ac454646fca4e61861dbed05d5', 'request_id': 'req-092ee285-b29d-4d9f-a03a-b2bb07294606'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.337983 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:28.350571 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] The token cache is being invalidate because project ffd072ac454646fca4e61861dbed05d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:28.350957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.351698 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.361703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1333/2699] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:28 2022] DELETE /identity/v3/projects/ffd072ac454646fca4e61861dbed05d5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:28.361703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.362429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:28.376319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c8a19-a768-4443-9214-dcabc6f4d7bb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.396554 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:28.474229 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ee285-b29d-4d9f-a03a-b2bb07294606 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.475025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.475735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.476077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.477050 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.477249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.480527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.483149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.484130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.484463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.484757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] PATH_INFO: `/v3/users/9458b797aba24c068e137a1e34520d3b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.486071 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9458b797aba24c068e137a1e34520d3b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.488054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.518890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1367/2700] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:28 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:28.523752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.526639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:28.544852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600077cb-1c4b-418b-baea-dd517079030b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.550639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.551758 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.552052 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.552382 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.553284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.555013 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9458b797aba24c068e137a1e34520d3b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.555570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.565887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9458b797aba24c068e137a1e34520d3b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.568193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1368/2701] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:28.572537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.583704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9458b797aba24c068e137a1e34520d3b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.584794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9458b797aba24c068e137a1e34520d3b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.586129 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d180b-356b-45f5-97f5-ea67c7f78642 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.597814 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9458b797aba24c068e137a1e34520d3b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:28.602527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.605529 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.606182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.606444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] PATH_INFO: `/v3/projects/749fc1751baa49f485a7abdcc0d4a304/users/e99dd22e8a3e4fb4ba69169a4accec04/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.608980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1334/2702] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:28 2022] DELETE /identity/v3/users/9458b797aba24c068e137a1e34520d3b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:28.635057 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749fc1751baa49f485a7abdcc0d4a304, user_id=e99dd22e8a3e4fb4ba69169a4accec04, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.638068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.660282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1369/2703] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:28 2022] PUT /identity/v3/projects/749fc1751baa49f485a7abdcc0d4a304/users/e99dd22e8a3e4fb4ba69169a4accec04/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:28.664174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.665923 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:28.754077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d49f5-c7b4-4896-bc66-2f28a0a6dae1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.806763 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:28.831345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea73665-47c5-4a2a-8007-9b0969d84af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.831967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73665-47c5-4a2a-8007-9b0969d84af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.832312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73665-47c5-4a2a-8007-9b0969d84af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.832870 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73665-47c5-4a2a-8007-9b0969d84af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.833817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea73665-47c5-4a2a-8007-9b0969d84af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.835419 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea73665-47c5-4a2a-8007-9b0969d84af3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.846169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1335/2704] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:28.850556 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.864386 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a303a-40ee-442b-8068-28d431cfccc7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:28.871546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:28.872324 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.872740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.873086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] PATH_INFO: `/v3/projects/749fc1751baa49f485a7abdcc0d4a304/users/e99dd22e8a3e4fb4ba69169a4accec04/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.875453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=749fc1751baa49f485a7abdcc0d4a304, user_id=e99dd22e8a3e4fb4ba69169a4accec04, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:28.878297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:28.897714 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1370/2705] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:28 2022] PUT /identity/v3/projects/749fc1751baa49f485a7abdcc0d4a304/users/e99dd22e8a3e4fb4ba69169a4accec04/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:28.906770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:28.907056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:28.907352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:28.942692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] MFA Rules not processed for user `e99dd22e8a3e4fb4ba69169a4accec04`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:28.943936 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:28.985896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:28.985896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:29.065168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1336/2706] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:29.074328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef9b9-a31f-4ef8-b02c-4ad3a81da6fc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.076692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:29.078089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:29.079576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.080241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.080600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.080801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] PATH_INFO: `/v3/projects/aabae08227a44e2ea99e8603e96de99a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.081948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabae08227a44e2ea99e8603e96de99a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.084252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.123954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:29.151857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabae08227a44e2ea99e8603e96de99a', 'request_id': 'req-907a6c10-d898-46fd-b7e8-996680807ed7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:29.159243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a0e5a-4bbd-4035-be26-1c3b3efecb86 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.164064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabae08227a44e2ea99e8603e96de99a', 'request_id': 'req-907a6c10-d898-46fd-b7e8-996680807ed7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:29.174535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabae08227a44e2ea99e8603e96de99a', 'request_id': 'req-907a6c10-d898-46fd-b7e8-996680807ed7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:29.202240 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] The token cache is being invalidate because project aabae08227a44e2ea99e8603e96de99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:29.202732 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:29.203547 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-907a6c10-d898-46fd-b7e8-996680807ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:29.205266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1371/2707] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:28 2022] DELETE /identity/v3/projects/aabae08227a44e2ea99e8603e96de99a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:29.210746 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f503194-887d-41fd-afad-ec5fa1bc283b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.210746 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f503194-887d-41fd-afad-ec5fa1bc283b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.211145 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f503194-887d-41fd-afad-ec5fa1bc283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.233397 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9f503194-887d-41fd-afad-ec5fa1bc283b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:29.234675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9f503194-887d-41fd-afad-ec5fa1bc283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:29.243835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.244531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.244824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.245099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.247171 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:29.298633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:29.351832 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1372/2708] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:29.356589 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.358119 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.361239 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af7b7a-beea-45c1-87db-ad7551c3b8a5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.361931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af7b7a-beea-45c1-87db-ad7551c3b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.362549 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af7b7a-beea-45c1-87db-ad7551c3b8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.372145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1337/2709] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:29.406008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-34af7b7a-beea-45c1-87db-ad7551c3b8a5 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:29.407526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34af7b7a-beea-45c1-87db-ad7551c3b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:29.416583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1373/2710] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:29.420719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:29.433008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e351c7-5ace-4b1a-935c-59ddb4863af4 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.439298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.439950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.440237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.440542 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.441169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:29.441467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.443400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.466394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1338/2711] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:29.471318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af7b7a-beea-45c1-87db-ad7551c3b8a5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:29.485037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af7b7a-beea-45c1-87db-ad7551c3b8a5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.492089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.492692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.492972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.493250 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.502125 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.504445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.557962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1374/2712] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:29 2022] POST /identity/v3/projects => generated 397 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:29.559443 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:29.571314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c8954-5254-4497-9f77-6e512a60c325 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.647682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.648283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.648566 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.648842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.649495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.651505 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.682426 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1339/2713] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:29 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:29.686708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:29.697840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb704eb-d72c-4db2-a76a-39ced2bd615e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.703997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.704602 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.704875 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.705137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.705818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.708171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.721263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1375/2714] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:29.725348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:29.737603 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662ea1-3eab-40d0-8812-5514431a9f77 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.744093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fb489-f951-42f8-b638-1d376a6c420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.744747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fb489-f951-42f8-b638-1d376a6c420e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.745089 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fb489-f951-42f8-b638-1d376a6c420e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.745481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fb489-f951-42f8-b638-1d376a6c420e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.746217 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fb489-f951-42f8-b638-1d376a6c420e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.747843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fb489-f951-42f8-b638-1d376a6c420e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.760825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1340/2715] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:29.765263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:29.778526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809add8-e630-4e7d-a037-38989c8f35a1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:29.783372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:29.784043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.784315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.784585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] PATH_INFO: `/v3/projects/422f51e05fe641899b38b2619bfac76f/users/c9f5ccb7e7e3486281fccc6136c07507/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.802336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] RBAC: Authorizing `identity:create_grant(project_id=422f51e05fe641899b38b2619bfac76f, user_id=c9f5ccb7e7e3486281fccc6136c07507, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:29.805025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:29.827390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1376/2716] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:29 2022] PUT /identity/v3/projects/422f51e05fe641899b38b2619bfac76f/users/c9f5ccb7e7e3486281fccc6136c07507/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:29.903965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:29.904252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:29.904533 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:29.950596 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] MFA Rules not processed for user `c9f5ccb7e7e3486281fccc6136c07507`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:29.951738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:30.047954 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1341/2717] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:29 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:30.063630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:30.068209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:30.117739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:30.119385 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:30.136759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6482fc6-606b-41bd-bb87-0795086e059d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:30.141192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:30.142155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:30.142440 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:30.142856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:30.150305 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:30.150993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:30.153707 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:30.155411 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:30.173479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1377/2718] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:30 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:30.208308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c48db9-0fbc-4fcd-b7f0-ccaea4c969a1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:30.210678 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:30.211646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:30.212120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:30.212544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:30.215451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:30.295173 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:30.296661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:30.325451 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:30.327306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:30.346916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1342/2719] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:30.410818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a9d64-b0bf-4eeb-88e4-e1cf8ca66527 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:30.417126 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:30.417986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:30.418502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:30.418779 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:30.420668 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:30.434750 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:30.436837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:30.448152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1378/2720] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:40.338980 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=38 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:40.443026 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=34 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:41.050741 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:41.060972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6d189-c7cb-47da-acfe-259b5fa1c86c service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:41.064873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf8bf-f971-40f8-b34a-623dcc2e163d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:41.065573 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf8bf-f971-40f8-b34a-623dcc2e163d service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:41.065907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf8bf-f971-40f8-b34a-623dcc2e163d service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:41.066181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf8bf-f971-40f8-b34a-623dcc2e163d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:41.068130 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-642cf8bf-f971-40f8-b34a-623dcc2e163d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:41.136710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf8bf-f971-40f8-b34a-623dcc2e163d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:41.138196 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf8bf-f971-40f8-b34a-623dcc2e163d service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:41.150334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1343/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:42.738456 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:42.832841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd38a-963f-4b32-b1c9-2c4733d9c04c service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:42.837896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:42.838652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:42.838932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:42.839209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] PATH_INFO: `/v3/users/c9f5ccb7e7e3486281fccc6136c07507` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:42.840634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f5ccb7e7e3486281fccc6136c07507)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:42.842640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:42.901076 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f5ccb7e7e3486281fccc6136c07507'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:42.913731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f5ccb7e7e3486281fccc6136c07507'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:42.926027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f5ccb7e7e3486281fccc6136c07507'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:42.926815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f5ccb7e7e3486281fccc6136c07507'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:42.935941 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f5ccb7e7e3486281fccc6136c07507'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:42.943669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1379/2722] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:42 2022] DELETE /identity/v3/users/c9f5ccb7e7e3486281fccc6136c07507 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:42.948402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:42.949091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:42.949091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:42.970206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:42.971738 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.055017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1344/2723] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:43.067046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:43.069043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.140451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7935d-b19b-4215-a823-46dcc60f5044 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:43.146195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:43.146982 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.147288 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.147566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.158776 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:43.160346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:43.172829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1380/2724] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:43.506823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:43.508477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.521605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:43.524185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.524931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df78164-1f1d-426e-881d-64c0f1113f4f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:43.531286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:43.532062 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.532406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.532740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] PATH_INFO: `/v3/users/195b777b82194a0f879dc706afc9c89b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.534142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=195b777b82194a0f879dc706afc9c89b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:43.536144 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:43.542539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4ebe-3cf4-4ea5-a667-3e179344dd46 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:43.549230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:43.549990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.550462 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.550757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] PATH_INFO: `/v3/projects/422f51e05fe641899b38b2619bfac76f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.552533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=422f51e05fe641899b38b2619bfac76f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:43.554091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:43.578266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:43.599188 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '195b777b82194a0f879dc706afc9c89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.609489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '195b777b82194a0f879dc706afc9c89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.616564 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '422f51e05fe641899b38b2619bfac76f', 'request_id': 'req-50de2035-f078-4a23-ab1c-a13bf25d50ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.623552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '422f51e05fe641899b38b2619bfac76f', 'request_id': 'req-50de2035-f078-4a23-ab1c-a13bf25d50ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.624685 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '195b777b82194a0f879dc706afc9c89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.625490 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '195b777b82194a0f879dc706afc9c89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.633084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '422f51e05fe641899b38b2619bfac76f', 'request_id': 'req-50de2035-f078-4a23-ab1c-a13bf25d50ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.633295 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a3b36348-73ab-4561-a875-8920152f1929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '195b777b82194a0f879dc706afc9c89b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.639722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1345/2725] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:43 2022] DELETE /identity/v3/users/195b777b82194a0f879dc706afc9c89b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:43.644943 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb632a-54f6-41e5-9a26-a5e564b99a35 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.645231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb632a-54f6-41e5-9a26-a5e564b99a35 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.645508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb632a-54f6-41e5-9a26-a5e564b99a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.645737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] The token cache is being invalidate because project 422f51e05fe641899b38b2619bfac76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:43.646036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.646679 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-50de2035-f078-4a23-ab1c-a13bf25d50ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:43.651638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1381/2726] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:43 2022] DELETE /identity/v3/projects/422f51e05fe641899b38b2619bfac76f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:43.653005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.653340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.653741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.666435 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e7fb632a-54f6-41e5-9a26-a5e564b99a35 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:43.667700 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e7fb632a-54f6-41e5-9a26-a5e564b99a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.673365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:43.674322 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:43.675595 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.735097 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1346/2727] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:43.737097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1382/2728] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:43.745232 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:43.745928 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ced77-3ec5-4b2e-9a85-2e4b64a2eeed None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.746049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ced77-3ec5-4b2e-9a85-2e4b64a2eeed None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.746307 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ced77-3ec5-4b2e-9a85-2e4b64a2eeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.746813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.766975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fd2ced77-3ec5-4b2e-9a85-2e4b64a2eeed None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:43.768646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fd2ced77-3ec5-4b2e-9a85-2e4b64a2eeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:43.780149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1347/2729] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:43.781224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ced77-3ec5-4b2e-9a85-2e4b64a2eeed None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:43.790916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ced77-3ec5-4b2e-9a85-2e4b64a2eeed None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:43.795639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:43.796252 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.796527 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.796797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.797444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:43.797786 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:43.799912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:43.808294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c328a6-a30a-487f-ac71-e475428f556b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:43.812096 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:43.812538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1348/2730] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:19:43.812717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.812998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.813271 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.816875 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:43.824032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:43.825785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:43.827686 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186c5b-efcd-4bee-b88b-0e9d5b59ca76 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:43.835239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:43.835863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.836148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.836416 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.837054 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:43.838068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1383/2731] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:43.839006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:43.870688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1349/2732] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:43 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:43.875429 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:43.887448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fb0d2-06b1-43da-9fcf-7890b1eb4cef service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:43.940753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:43.958329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:43.958912 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:43.959329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:43.959601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:43.960262 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:43.962685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:43.994652 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1384/2733] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:43 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:43.995896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:44.008222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83021963-d49c-475d-bdbb-8df17d70f323 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:44.013575 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:44.014239 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.014528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.014782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.015425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:44.017090 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:44.030135 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1350/2734] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:44.034352 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:44.044968 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09fa1e-3682-4bb1-acca-f789f2f928e7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:44.050326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e539b-4d9d-42f4-b6e9-366d85ebe0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:44.050948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e539b-4d9d-42f4-b6e9-366d85ebe0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.051226 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e539b-4d9d-42f4-b6e9-366d85ebe0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.051494 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e539b-4d9d-42f4-b6e9-366d85ebe0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.052139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e539b-4d9d-42f4-b6e9-366d85ebe0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:44.053781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e539b-4d9d-42f4-b6e9-366d85ebe0e7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:44.064377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1385/2735] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:44.068959 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:44.079459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c8fc4-f316-429e-954a-e2291f2bfef4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:44.084653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:44.085415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.085846 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.086126 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] PATH_INFO: `/v3/projects/5f20e431b2b140ebb7c895ebd5954f67/users/3c5f2dea7bab4b83b09c0c61b21e9700/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.104582 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f20e431b2b140ebb7c895ebd5954f67, user_id=3c5f2dea7bab4b83b09c0c61b21e9700, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:44.107621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:44.124138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1351/2736] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:44 2022] PUT /identity/v3/projects/5f20e431b2b140ebb7c895ebd5954f67/users/3c5f2dea7bab4b83b09c0c61b21e9700/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:44.132576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b955da6-3262-4501-9077-f407c34c718c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.132576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b955da6-3262-4501-9077-f407c34c718c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.132576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b955da6-3262-4501-9077-f407c34c718c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.134228 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:44.136168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:44.160657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8b955da6-3262-4501-9077-f407c34c718c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:44.162621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8b955da6-3262-4501-9077-f407c34c718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:44.209283 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5e59e-78ed-41b5-bb48-8f2c06951ab4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:44.214682 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:44.215722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.215722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.215949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] PATH_INFO: `/v3/projects/2da1efd0c041487488811ff5bc1f77e3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.217183 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da1efd0c041487488811ff5bc1f77e3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:44.219454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:44.228855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1386/2737] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:44.238441 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b955da6-3262-4501-9077-f407c34c718c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:44.239932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8b955da6-3262-4501-9077-f407c34c718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:44.262857 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:44.275950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da1efd0c041487488811ff5bc1f77e3', 'request_id': 'req-ec7370ef-5c74-48d3-a918-87d0a01c10bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.281588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da1efd0c041487488811ff5bc1f77e3', 'request_id': 'req-ec7370ef-5c74-48d3-a918-87d0a01c10bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.288262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da1efd0c041487488811ff5bc1f77e3', 'request_id': 'req-ec7370ef-5c74-48d3-a918-87d0a01c10bd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.303001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b955da6-3262-4501-9077-f407c34c718c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:44.303287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] The token cache is being invalidate because project 2da1efd0c041487488811ff5bc1f77e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:44.303576 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.304320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.305887 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1352/2738] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:44 2022] DELETE /identity/v3/projects/2da1efd0c041487488811ff5bc1f77e3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:44.309812 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:44.311187 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:44.355434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:44.356030 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.356302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.356570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.358395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:44.381012 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7370ef-5c74-48d3-a918-87d0a01c10bd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:44.387400 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:44.388082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.388361 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.388633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] PATH_INFO: `/v3/users/da65a2e4e1764868a8677ecaf05c3d85` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.389952 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da65a2e4e1764868a8677ecaf05c3d85)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:44.392030 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:44.415656 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:44.451872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:44.453417 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:44.456250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da65a2e4e1764868a8677ecaf05c3d85'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.465139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da65a2e4e1764868a8677ecaf05c3d85'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.465402 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1387/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:44.479503 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da65a2e4e1764868a8677ecaf05c3d85'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.479977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da65a2e4e1764868a8677ecaf05c3d85'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.488696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da65a2e4e1764868a8677ecaf05c3d85'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:44.496328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1353/2740] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:44 2022] DELETE /identity/v3/users/da65a2e4e1764868a8677ecaf05c3d85 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:44.809400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:44.822420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894ca15-26b2-44e1-b1e5-4bd2a737e578 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:44.927670 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:44.928811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:44.928811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:44.929121 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] PATH_INFO: `/v3/projects/3e618023d22348838e7427f816036de9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:44.930533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e618023d22348838e7427f816036de9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:44.932706 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:44.998572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e618023d22348838e7427f816036de9', 'request_id': 'req-03935812-c29e-4fa9-b5ad-34e9c1a18c39'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.005820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e618023d22348838e7427f816036de9', 'request_id': 'req-03935812-c29e-4fa9-b5ad-34e9c1a18c39'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.014171 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e618023d22348838e7427f816036de9', 'request_id': 'req-03935812-c29e-4fa9-b5ad-34e9c1a18c39'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.026629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] The token cache is being invalidate because project 3e618023d22348838e7427f816036de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:45.026925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.027585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.029089 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1388/2741] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:44 2022] DELETE /identity/v3/projects/3e618023d22348838e7427f816036de9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:45.033269 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:45.034768 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:45.048977 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223bfe7-47b8-4916-93b8-909ff62f83a1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:45.110779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:45.130315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:45.131131 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:45.131474 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:45.131729 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] PATH_INFO: `/v3/users/e99dd22e8a3e4fb4ba69169a4accec04` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:45.132933 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99dd22e8a3e4fb4ba69169a4accec04)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:45.134884 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:45.201124 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99dd22e8a3e4fb4ba69169a4accec04'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.213222 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99dd22e8a3e4fb4ba69169a4accec04'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.232687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99dd22e8a3e4fb4ba69169a4accec04'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.233541 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99dd22e8a3e4fb4ba69169a4accec04'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.244041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3acf5666-0cfe-44db-8f8c-b1e5bdfbcd8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99dd22e8a3e4fb4ba69169a4accec04'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.251691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1354/2742] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:45 2022] DELETE /identity/v3/users/e99dd22e8a3e4fb4ba69169a4accec04 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:45.763527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:45.765427 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:45.787470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03935812-c29e-4fa9-b5ad-34e9c1a18c39 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:45.885887 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:45.886612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:45.886949 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:45.887221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] PATH_INFO: `/v3/projects/749fc1751baa49f485a7abdcc0d4a304` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:45.888552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=749fc1751baa49f485a7abdcc0d4a304)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:45.890755 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:45.902785 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:45.964585 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749fc1751baa49f485a7abdcc0d4a304', 'request_id': 'req-873547cc-07a6-4b1c-a537-da40f381f5bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.976632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749fc1751baa49f485a7abdcc0d4a304', 'request_id': 'req-873547cc-07a6-4b1c-a537-da40f381f5bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:45.990701 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749fc1751baa49f485a7abdcc0d4a304', 'request_id': 'req-873547cc-07a6-4b1c-a537-da40f381f5bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:46.009455 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] The token cache is being invalidate because project 749fc1751baa49f485a7abdcc0d4a304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:46.010249 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:46.011581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-873547cc-07a6-4b1c-a537-da40f381f5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:46.013828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1389/2743] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:45 2022] DELETE /identity/v3/projects/749fc1751baa49f485a7abdcc0d4a304 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:46.037378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f34fb-7e1e-4d4c-bb69-6d371b0b6650 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.038425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f34fb-7e1e-4d4c-bb69-6d371b0b6650 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.038583 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f34fb-7e1e-4d4c-bb69-6d371b0b6650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.077154 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b74f34fb-7e1e-4d4c-bb69-6d371b0b6650 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:46.079640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b74f34fb-7e1e-4d4c-bb69-6d371b0b6650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:46.215612 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1355/2744] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:46.225442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45421c-a205-4c35-a9c8-30f7705189d3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.225797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45421c-a205-4c35-a9c8-30f7705189d3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.226060 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45421c-a205-4c35-a9c8-30f7705189d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.259127 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4c45421c-a205-4c35-a9c8-30f7705189d3 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:46.260613 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4c45421c-a205-4c35-a9c8-30f7705189d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:46.285370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1390/2745] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:46.287159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f34fb-7e1e-4d4c-bb69-6d371b0b6650 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:46.326831 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f34fb-7e1e-4d4c-bb69-6d371b0b6650 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:46.334224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:46.335031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.335751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.336234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.337036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:46.337348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:46.348941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:46.367807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1356/2746] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:46.373919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45421c-a205-4c35-a9c8-30f7705189d3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:46.394007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45421c-a205-4c35-a9c8-30f7705189d3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:46.407827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:46.409984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.410369 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.410539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.411242 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:46.413316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:46.457638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1391/2747] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:46 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:46.462320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:46.489622 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6cdac-c526-4127-8e3b-417594153fc2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:46.569710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:46.584146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:46.584776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.585100 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.585631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.586637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:46.588820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:46.618073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1357/2748] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:46 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:46.619625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:46.632300 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33548cd-a33a-42f0-b460-5ceb78ee5658 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:46.638692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:46.639424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.639716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.640028 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.640661 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:46.642593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:46.657993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1392/2749] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:46.659395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:46.670498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2f53d-0888-493d-a273-2eb35c045fe4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:46.675828 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f6bf5-7280-4a76-a400-db31d4da52ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:46.676515 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6bf5-7280-4a76-a400-db31d4da52ad admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.676888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6bf5-7280-4a76-a400-db31d4da52ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.677214 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6bf5-7280-4a76-a400-db31d4da52ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.677955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f6bf5-7280-4a76-a400-db31d4da52ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:46.679613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f6bf5-7280-4a76-a400-db31d4da52ad admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:46.692480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1358/2750] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:46.696687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:46.715641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be9a95-0ded-4be0-9152-0aa5f2c0ad1f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:46.720638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:46.721459 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.721825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.722107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] PATH_INFO: `/v3/projects/287531fe0100441ca51ac7aa946aaa71/users/3f243f5d61754436acd38c8f0be27495/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.738624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] RBAC: Authorizing `identity:create_grant(project_id=287531fe0100441ca51ac7aa946aaa71, user_id=3f243f5d61754436acd38c8f0be27495, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:46.741302 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:46.758737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1393/2751] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:46 2022] PUT /identity/v3/projects/287531fe0100441ca51ac7aa946aaa71/users/3f243f5d61754436acd38c8f0be27495/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:46.770438 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7c444-0c9d-48a4-8e81-fe8ad4fb539e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:46.770727 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7c444-0c9d-48a4-8e81-fe8ad4fb539e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:46.771000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7c444-0c9d-48a4-8e81-fe8ad4fb539e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:46.803310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-17c7c444-0c9d-48a4-8e81-fe8ad4fb539e None None] MFA Rules not processed for user `3f243f5d61754436acd38c8f0be27495`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:46.804817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17c7c444-0c9d-48a4-8e81-fe8ad4fb539e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:46.904318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1359/2752] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:46.910107 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:47.022318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc282c7-69bb-4f13-9dd4-33595531d60c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:47.035846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:47.036717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.037027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.037395 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:47.038235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:47.040504 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:47.076618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1394/2753] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:46 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:47.081537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7c444-0c9d-48a4-8e81-fe8ad4fb539e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:47.094424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7c444-0c9d-48a4-8e81-fe8ad4fb539e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:47.174334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:47.178660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.179002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.179286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:47.181784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:47.188021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:47.220993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1360/2754] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:47 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:47.223869 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:47.238596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacc28d-833c-4608-a8ca-49aa983cf781 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:47.246784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:47.247483 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.247770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.248046 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:47.248703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:47.250450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:47.274889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:47.282392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1395/2755] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:47.289800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444d1d0-66d8-4857-a31d-03e83073985d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:47.297833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:47.299128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.299262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.299472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] PATH_INFO: `/v3/projects/238bbdaa067944ce81c5320787fd6a3f/users/7a6b73287109497692acb374fa134ab7/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:47.328276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] RBAC: Authorizing `identity:create_grant(project_id=238bbdaa067944ce81c5320787fd6a3f, user_id=7a6b73287109497692acb374fa134ab7, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:47.332638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:47.358355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1361/2756] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:47 2022] PUT /identity/v3/projects/238bbdaa067944ce81c5320787fd6a3f/users/7a6b73287109497692acb374fa134ab7/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:47.362356 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:47.373590 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7afaca-d504-427c-b251-72e9d816b253 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:47.374781 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:47.509093 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d164e0-7f1f-43a3-8c55-adec3176977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:47.510051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d164e0-7f1f-43a3-8c55-adec3176977b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.510424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d164e0-7f1f-43a3-8c55-adec3176977b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.510705 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d164e0-7f1f-43a3-8c55-adec3176977b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:47.511384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d164e0-7f1f-43a3-8c55-adec3176977b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:47.513967 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d164e0-7f1f-43a3-8c55-adec3176977b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:47.537492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1396/2757] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:47.544300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:47.580351 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a01a45-6a0b-4137-b586-55e13cb0994e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:47.592537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:47.593310 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.594058 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.594344 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] PATH_INFO: `/v3/projects/238bbdaa067944ce81c5320787fd6a3f/users/7a6b73287109497692acb374fa134ab7/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:47.599368 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=238bbdaa067944ce81c5320787fd6a3f, user_id=7a6b73287109497692acb374fa134ab7, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:47.605338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:47.640208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1362/2758] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:47 2022] PUT /identity/v3/projects/238bbdaa067944ce81c5320787fd6a3f/users/7a6b73287109497692acb374fa134ab7/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:47.647444 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc06b3-086f-4bad-ac31-a1c1b1acbc30 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.647922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc06b3-086f-4bad-ac31-a1c1b1acbc30 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.648520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc06b3-086f-4bad-ac31-a1c1b1acbc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:47.717203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2ffc06b3-086f-4bad-ac31-a1c1b1acbc30 None None] MFA Rules not processed for user `7a6b73287109497692acb374fa134ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:47.718540 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2ffc06b3-086f-4bad-ac31-a1c1b1acbc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:47.822262 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1397/2759] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:47.835684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:47.838120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:47.868417 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:47.938757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b84b79-c82d-4c50-a0aa-89cccf7f8676 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:47.945534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:47.946202 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:47.946480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:47.946752 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:48.023481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:48.025436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:48.038206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1363/2760] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:49.729392 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323812-0c32-450c-8b51-04bc88cafc81 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:49.729931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323812-0c32-450c-8b51-04bc88cafc81 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:49.730212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e323812-0c32-450c-8b51-04bc88cafc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:49.770708 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5e323812-0c32-450c-8b51-04bc88cafc81 None None] MFA Rules not processed for user `3c5f2dea7bab4b83b09c0c61b21e9700`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:49.772181 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5e323812-0c32-450c-8b51-04bc88cafc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:49.860233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1398/2761] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:49.866973 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:49.868819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:49.974181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11cb23-36fe-4abe-9cd1-a9316ac77105 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:49.979177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:49.979799 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:49.980083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:49.980349 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:49.982298 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:49.984243 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:50.019888 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1364/2762] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:49 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:50.024345 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e323812-0c32-450c-8b51-04bc88cafc81 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:50.036486 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e323812-0c32-450c-8b51-04bc88cafc81 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:50.054955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:50.124588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:50.125216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:50.126012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:50.126298 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:50.127085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:50.129268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:50.167892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1399/2763] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:50 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:50.171703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:50.192989 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33c61c-294b-40f4-9802-0096962b118f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:50.202919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:50.203886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:50.204424 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:50.205237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:50.206594 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:50.211666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:50.229037 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1365/2764] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:50.234833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:50.245735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa34350-b33f-43d8-a8ad-81b683923aca admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:50.250628 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:50.251309 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:50.251683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:50.252042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] PATH_INFO: `/v3/projects/ae25b3c0c8d5464d8779cb8bdc03cc75/users/2f45f33b0f254a9e8927e1153bb36617/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:50.270930 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae25b3c0c8d5464d8779cb8bdc03cc75, user_id=2f45f33b0f254a9e8927e1153bb36617, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:50.274029 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:50.293971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1400/2765] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:50 2022] PUT /identity/v3/projects/ae25b3c0c8d5464d8779cb8bdc03cc75/users/2f45f33b0f254a9e8927e1153bb36617/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:50.298088 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:50.312643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc1db-f146-46ee-b404-44255548b7fc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:50.372526 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:50.422495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:50.423195 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:50.423606 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:50.423942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:50.424684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:50.426467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:50.439976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1366/2766] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:50.447835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:50.459157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb230e-ea6b-4eed-9133-0d248342a924 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:50.466933 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47734b11-7210-421d-b9e7-74e3d4981e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:50.467615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47734b11-7210-421d-b9e7-74e3d4981e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:50.467890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47734b11-7210-421d-b9e7-74e3d4981e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:50.468270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47734b11-7210-421d-b9e7-74e3d4981e05 admin admin] PATH_INFO: `/v3/projects/ae25b3c0c8d5464d8779cb8bdc03cc75/users/2f45f33b0f254a9e8927e1153bb36617/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:50.471235 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47734b11-7210-421d-b9e7-74e3d4981e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae25b3c0c8d5464d8779cb8bdc03cc75, user_id=2f45f33b0f254a9e8927e1153bb36617, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:50.475523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47734b11-7210-421d-b9e7-74e3d4981e05 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:50.489188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1401/2767] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:50 2022] PUT /identity/v3/projects/ae25b3c0c8d5464d8779cb8bdc03cc75/users/2f45f33b0f254a9e8927e1153bb36617/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:51.860586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:51.877461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbd0e4-9038-4650-a1ff-b76c8a5fbbc0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:51.968158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:51.969256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:51.969715 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:51.970161 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] PATH_INFO: `/v3/users/3f243f5d61754436acd38c8f0be27495` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:51.971780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f243f5d61754436acd38c8f0be27495)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:51.974732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:52.044804 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f243f5d61754436acd38c8f0be27495'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:52.056905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f243f5d61754436acd38c8f0be27495'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:52.076020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f243f5d61754436acd38c8f0be27495'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:52.076834 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f243f5d61754436acd38c8f0be27495'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:52.095931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f243f5d61754436acd38c8f0be27495'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:52.110402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1367/2768] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:51 2022] DELETE /identity/v3/users/3f243f5d61754436acd38c8f0be27495 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:52.113243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:52.113815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:52.113815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:52.142430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:52.144973 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:52.225263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1402/2769] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:52.235960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:52.238211 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:52.366105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da72d-ba92-4e46-9c5d-3a4d2af43f6f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:52.372517 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:52.373236 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:52.373540 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:52.373924 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:52.403819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:52.405332 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:52.410712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:52.441794 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1368/2770] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:52.883619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:52.885489 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:52.906247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5977c-ec80-4dde-a992-2f3caac6dcfe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:52.915442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:52.916209 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:52.917259 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:52.917639 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] PATH_INFO: `/v3/projects/287531fe0100441ca51ac7aa946aaa71` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:52.919638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=287531fe0100441ca51ac7aa946aaa71)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:52.922998 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:53.003618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '287531fe0100441ca51ac7aa946aaa71', 'request_id': 'req-02685789-2de9-4f7f-8ec4-178f12e6b91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.016177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '287531fe0100441ca51ac7aa946aaa71', 'request_id': 'req-02685789-2de9-4f7f-8ec4-178f12e6b91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.040328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '287531fe0100441ca51ac7aa946aaa71', 'request_id': 'req-02685789-2de9-4f7f-8ec4-178f12e6b91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.045211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:53.065704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] The token cache is being invalidate because project 287531fe0100441ca51ac7aa946aaa71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:53.066482 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.070655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.083114 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1403/2771] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:52 2022] DELETE /identity/v3/projects/287531fe0100441ca51ac7aa946aaa71 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:53.085177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:53.096218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de02590-f077-448f-8f73-861c331ad170 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:53.217247 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:53.218024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:53.218314 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:53.218796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] PATH_INFO: `/v3/users/7a6b73287109497692acb374fa134ab7` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:53.220085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6b73287109497692acb374fa134ab7)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:53.223216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:53.303261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6b73287109497692acb374fa134ab7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.314774 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6b73287109497692acb374fa134ab7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.337572 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6b73287109497692acb374fa134ab7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.339288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6b73287109497692acb374fa134ab7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.355431 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6b73287109497692acb374fa134ab7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:53.363731 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1369/2772] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:53 2022] DELETE /identity/v3/users/7a6b73287109497692acb374fa134ab7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:53.951408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:53.953698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:54.013174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02685789-2de9-4f7f-8ec4-178f12e6b91f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:54.126911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:54.127373 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:54.127653 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:54.127929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] PATH_INFO: `/v3/projects/238bbdaa067944ce81c5320787fd6a3f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:54.129234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=238bbdaa067944ce81c5320787fd6a3f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:54.132999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:54.307450 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238bbdaa067944ce81c5320787fd6a3f', 'request_id': 'req-8f099a1f-ca7b-45ae-b386-3c925b11f359'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.340592 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238bbdaa067944ce81c5320787fd6a3f', 'request_id': 'req-8f099a1f-ca7b-45ae-b386-3c925b11f359'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.354162 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238bbdaa067944ce81c5320787fd6a3f', 'request_id': 'req-8f099a1f-ca7b-45ae-b386-3c925b11f359'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.363538 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:54.367095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:54.390916 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] The token cache is being invalidate because project 238bbdaa067944ce81c5320787fd6a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:54.391497 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.393451 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f099a1f-ca7b-45ae-b386-3c925b11f359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.394075 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75809e05-abe8-423c-adcb-0939de0bd123 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:54.395316 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1404/2773] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:53 2022] DELETE /identity/v3/projects/238bbdaa067944ce81c5320787fd6a3f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:54.407038 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3605693-758e-4b96-aebc-f093f3aef320 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:54.407443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3605693-758e-4b96-aebc-f093f3aef320 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:54.407919 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3605693-758e-4b96-aebc-f093f3aef320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:54.440307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c3605693-758e-4b96-aebc-f093f3aef320 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:54.443505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c3605693-758e-4b96-aebc-f093f3aef320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:54.571963 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:54.572837 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:54.573185 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:54.573534 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] PATH_INFO: `/v3/users/14f98e16cd3244ceb66904aa082ef9f2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:54.575399 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f98e16cd3244ceb66904aa082ef9f2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:54.577481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:54.614780 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:54.651115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1405/2774] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:54.673123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf73e5-1afe-45f3-92f7-addfeefcaff1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:54.673721 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf73e5-1afe-45f3-92f7-addfeefcaff1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:54.677653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f98e16cd3244ceb66904aa082ef9f2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.687575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf73e5-1afe-45f3-92f7-addfeefcaff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:54.697242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f98e16cd3244ceb66904aa082ef9f2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.729268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f98e16cd3244ceb66904aa082ef9f2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.731086 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f98e16cd3244ceb66904aa082ef9f2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.746509 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4baf73e5-1afe-45f3-92f7-addfeefcaff1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:54.748264 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4baf73e5-1afe-45f3-92f7-addfeefcaff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:54.752753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-64b8d79b-e373-41e1-8bfa-7b9b831d9244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f98e16cd3244ceb66904aa082ef9f2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:54.765899 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1370/2775] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:54 2022] DELETE /identity/v3/users/14f98e16cd3244ceb66904aa082ef9f2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:54.769699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1406/2776] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:54.777431 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf73e5-1afe-45f3-92f7-addfeefcaff1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:54.778364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:54.778749 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:54.779274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:54.794398 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf73e5-1afe-45f3-92f7-addfeefcaff1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:54.816395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:54.818867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:54.820567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:54.950191 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:54.951290 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:54.952280 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:54.956152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:54.957219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:54.958043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:54.966063 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:54.989408 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1407/2777] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:54.994471 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:55.005656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1371/2778] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:55.007241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:55.008819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:55.013049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c5473-b1f0-4af3-b592-23bbb01894da admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:55.030070 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:55.030904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:55.031357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:55.031873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:55.032811 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:55.035810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:55.131978 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1408/2779] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:54 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:55.140172 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:55.145003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de54871-e953-4b10-b12f-a2abbf901367 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:55.262238 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:55.263206 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:55.263688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:55.264348 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:55.278599 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:55.367013 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f0b2c-f260-457d-bd16-027bcf7e56f2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:55.380940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:55.381610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:55.381990 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:55.382453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:55.391508 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:55.393931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:55.417036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:55.421027 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:55.447702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1409/2781] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:55 2022] POST /identity/v3/users => generated 293 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:55.448079 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1372/2781] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:55.454020 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:55.490467 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365108d-70fb-4cd1-bf69-dcc1cc182fe4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:55.508027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:55.508846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:55.509428 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:55.510216 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:55.511184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:55.519733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:55.546199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1410/2782] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:55.551734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:55.573625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056e67e-20bb-496f-98a3-095db7652be0 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:55.582807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42226a61-775a-4dbf-bb66-5cb306601439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:55.583779 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42226a61-775a-4dbf-bb66-5cb306601439 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:55.584072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42226a61-775a-4dbf-bb66-5cb306601439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:55.584355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42226a61-775a-4dbf-bb66-5cb306601439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:55.585328 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42226a61-775a-4dbf-bb66-5cb306601439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:55.587352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42226a61-775a-4dbf-bb66-5cb306601439 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:55.618430 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1373/2783] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:55.627863 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:55.647012 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026ed2d-960f-42f9-8da3-9edfa97c3ac0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:55.671607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:55.672516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:55.672885 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:55.673297 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] PATH_INFO: `/v3/projects/0865a2a25a5f4901b689e5c660a57ba6/users/8b1fcf89e61b4f3fadb3af28310bc529/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:55.709637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0865a2a25a5f4901b689e5c660a57ba6, user_id=8b1fcf89e61b4f3fadb3af28310bc529, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:55.714214 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:55.765815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1411/2784] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:55 2022] PUT /identity/v3/projects/0865a2a25a5f4901b689e5c660a57ba6/users/8b1fcf89e61b4f3fadb3af28310bc529/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:55.777876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:55.778485 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:55.778867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:55.905907 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] MFA Rules not processed for user `8b1fcf89e61b4f3fadb3af28310bc529`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:55.911966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:56.051820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1374/2785] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:56.057357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:56.159705 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:56.161937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:56.168367 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:56.215764 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cef2a-3fe3-485b-b9f9-202fee51df82 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:56.229204 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:56.229984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.230276 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.230584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.231274 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:56.239154 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:56.319638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1412/2786] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:56 2022] POST /identity/v3/projects => generated 383 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:56.323717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:56.358458 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b02a93-f4bf-4ce0-a6d5-1044070e3d90 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:56.439703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:56.451405 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579974a2-fa6f-4569-925d-26114aff74b1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:56.470603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:56.471117 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.471453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.471672 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.472826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:56.475128 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:56.509960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:56.510681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.510967 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.511263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] PATH_INFO: `/v3/projects/11f1f3eefd684734900eb5e32a37bc85` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.515477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f1f3eefd684734900eb5e32a37bc85)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:56.518070 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:56.525841 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1413/2787] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:56 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:56.531001 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:56.546857 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba763f-fab7-45a6-b1ae-d9f765ecdaca admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:56.555818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:56.557119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.557695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.558243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.559956 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:56.563194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:56.584921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1414/2788] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:56.589086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:56.623643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2031d71-1e2b-481b-adc1-10b2dfd6f97f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:56.623863 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f1f3eefd684734900eb5e32a37bc85', 'request_id': 'req-97d5ac22-9726-421c-b536-9a2079902ee2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:56.630649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f1f3eefd684734900eb5e32a37bc85', 'request_id': 'req-97d5ac22-9726-421c-b536-9a2079902ee2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:56.641072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f1f3eefd684734900eb5e32a37bc85', 'request_id': 'req-97d5ac22-9726-421c-b536-9a2079902ee2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:56.660178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] The token cache is being invalidate because project 11f1f3eefd684734900eb5e32a37bc85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:56.660717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:56.661593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-97d5ac22-9726-421c-b536-9a2079902ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:56.665769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1375/2789] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:56 2022] DELETE /identity/v3/projects/11f1f3eefd684734900eb5e32a37bc85 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:56.679560 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888872-1f83-4150-960c-733b0dc30cba None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.679792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888872-1f83-4150-960c-733b0dc30cba None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.680078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888872-1f83-4150-960c-733b0dc30cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.726319 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-16888872-1f83-4150-960c-733b0dc30cba None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:56.728305 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-16888872-1f83-4150-960c-733b0dc30cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:56.757212 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:56.758185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.758655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.759094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.759942 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:56.770065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:56.793520 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1415/2790] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:56.798695 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:56.800778 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:56.815646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5be306-11bd-4e38-b9ac-b9c21b68f2c0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:56.821137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec552-1a5e-4e02-90c8-72ce4db9317c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:56.822008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec552-1a5e-4e02-90c8-72ce4db9317c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.822275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec552-1a5e-4e02-90c8-72ce4db9317c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.822550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec552-1a5e-4e02-90c8-72ce4db9317c admin admin] PATH_INFO: `/v3/projects/b57fc8e37bc44dc781534fe90d73cb67/users/8757a267268c48d0bbc266b10377b89a/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.842972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec552-1a5e-4e02-90c8-72ce4db9317c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57fc8e37bc44dc781534fe90d73cb67, user_id=8757a267268c48d0bbc266b10377b89a, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:56.846683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec552-1a5e-4e02-90c8-72ce4db9317c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:56.856475 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1376/2791] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:56.868200 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44326e-b2f2-432a-92c2-b710942be10d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.868518 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44326e-b2f2-432a-92c2-b710942be10d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.868889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44326e-b2f2-432a-92c2-b710942be10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.878237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1416/2792] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:56 2022] PUT /identity/v3/projects/b57fc8e37bc44dc781534fe90d73cb67/users/8757a267268c48d0bbc266b10377b89a/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:56.893385 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d8859-e424-4087-93dd-d319e14cb792 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:56.893650 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d8859-e424-4087-93dd-d319e14cb792 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:56.893988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d8859-e424-4087-93dd-d319e14cb792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:56.929311 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8d44326e-b2f2-432a-92c2-b710942be10d None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:56.931250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d44326e-b2f2-432a-92c2-b710942be10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:57.001064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-616d8859-e424-4087-93dd-d319e14cb792 None None] MFA Rules not processed for user `8757a267268c48d0bbc266b10377b89a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:57.004307 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-616d8859-e424-4087-93dd-d319e14cb792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:57.022630 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:57.080658 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:57.081066 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1377/2793] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:57.095957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44326e-b2f2-432a-92c2-b710942be10d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.125418 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44326e-b2f2-432a-92c2-b710942be10d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.136434 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70540-4c58-433f-b0f8-193e6469b711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.138620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70540-4c58-433f-b0f8-193e6469b711 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.139051 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70540-4c58-433f-b0f8-193e6469b711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.139425 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70540-4c58-433f-b0f8-193e6469b711 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.140251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70540-4c58-433f-b0f8-193e6469b711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:57.140648 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70540-4c58-433f-b0f8-193e6469b711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.143579 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70540-4c58-433f-b0f8-193e6469b711 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.177819 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1378/2794] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:57.179611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073f8f-7595-4cd3-babc-d389baa13911 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.179981 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073f8f-7595-4cd3-babc-d389baa13911 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.180205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073f8f-7595-4cd3-babc-d389baa13911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.211728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1417/2795] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:57.213051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d8859-e424-4087-93dd-d319e14cb792 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.230572 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d8859-e424-4087-93dd-d319e14cb792 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.231296 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-53073f8f-7595-4cd3-babc-d389baa13911 None None] MFA Rules not processed for user `2f45f33b0f254a9e8927e1153bb36617`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:57.234489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-53073f8f-7595-4cd3-babc-d389baa13911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:57.239002 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.240828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.241654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.243915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.245789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.250541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.292856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1418/2796] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:57 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:57.293713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.353851 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1379/2797] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:57.356003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073f8f-7595-4cd3-babc-d389baa13911 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.370462 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073f8f-7595-4cd3-babc-d389baa13911 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.416826 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb274d-466f-4359-8c7e-83472ff6b760 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.422760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.423495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.423840 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.424182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.424899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.426996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.473095 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.474166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.474486 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.474986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.476138 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.480550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1419/2798] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:57 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:57.480550 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.480821 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.482141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:57.533073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1380/2799] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:57 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:57.534122 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.545107 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0bff4-be89-48aa-abd7-47466017f03e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.588370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4d8c9-f93f-4823-9ca9-690f79cbaf26 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.594449 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.595932 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.596215 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.596542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.598669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:57.633277 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:57.633620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.634352 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.634642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.635031 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.635611 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.638472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.650337 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:57.671288 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1381/2800] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:57 2022] POST /identity/v3/users => generated 298 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:57.672073 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.687756 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.688226 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d02f8c-b2ff-42ca-aecc-3e9c78826815 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.689814 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.693919 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.694633 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.695127 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.695471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.696246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.698158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.700669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1420/2801] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:57.701527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.712646 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1382/2802] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:57.713581 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41927-8596-41a9-a3ba-629640df6b86 service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.722657 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.723824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.724478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.724844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.725112 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.726116 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.727899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.741507 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6f916-54c9-488b-b2a5-971fba33034f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.748629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1421/2803] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:57.749985 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.752331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.753256 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.753639 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.753986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.754770 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.756650 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.764065 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010c663-4621-424f-945e-d02caf2b613b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.769280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1383/2804] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:57.774911 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.775216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.776072 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.776364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.776689 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] PATH_INFO: `/v3/projects/a2832893d5ff4e66b94339c67e2370f3/users/a333e1dbf81c44c698b6af949ae2c5cb/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.787265 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458335a-13a0-4f6b-9c4b-a091f9d7cda1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:57.792808 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72373703-0b21-4731-a781-bf4649f6674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:57.793544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373703-0b21-4731-a781-bf4649f6674a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.793950 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373703-0b21-4731-a781-bf4649f6674a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.794261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373703-0b21-4731-a781-bf4649f6674a admin admin] PATH_INFO: `/v3/projects/d1a9b75e7c8a463a9d0c24483fbd6e0c/users/30dd191837464d258d563eac37683da5/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.810687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2832893d5ff4e66b94339c67e2370f3, user_id=a333e1dbf81c44c698b6af949ae2c5cb, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.816358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72373703-0b21-4731-a781-bf4649f6674a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a9b75e7c8a463a9d0c24483fbd6e0c, user_id=30dd191837464d258d563eac37683da5, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:57.816969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.818493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72373703-0b21-4731-a781-bf4649f6674a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:57.852718 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1422/2805] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:57 2022] PUT /identity/v3/projects/a2832893d5ff4e66b94339c67e2370f3/users/a333e1dbf81c44c698b6af949ae2c5cb/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:57.861117 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1384/2806] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:57 2022] PUT /identity/v3/projects/d1a9b75e7c8a463a9d0c24483fbd6e0c/users/30dd191837464d258d563eac37683da5/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:57.862541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:57.888746 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61379f-fed6-4e95-b435-e198dd601b03 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:57.889141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61379f-fed6-4e95-b435-e198dd601b03 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:57.889331 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61379f-fed6-4e95-b435-e198dd601b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:57.936048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1f61379f-fed6-4e95-b435-e198dd601b03 None None] MFA Rules not processed for user `30dd191837464d258d563eac37683da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:57.937338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1f61379f-fed6-4e95-b435-e198dd601b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:58.017827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcc40a-3767-4927-9208-4eadc0216e64 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:58.036199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1385/2807] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:58.038129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17b13a-cb74-4b52-9496-f09710dee298 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:58.038621 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17b13a-cb74-4b52-9496-f09710dee298 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:58.039102 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17b13a-cb74-4b52-9496-f09710dee298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:58.068467 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2b17b13a-cb74-4b52-9496-f09710dee298 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:58.070010 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b17b13a-cb74-4b52-9496-f09710dee298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:58.104780 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:58.105588 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:58.107254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:58.107254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:58.107254 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:58.108558 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:58.120139 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1423/2808] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:58.123759 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:58.125506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:58.147411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1386/2809] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:19:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:58.147942 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17b13a-cb74-4b52-9496-f09710dee298 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:58.160986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17b13a-cb74-4b52-9496-f09710dee298 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:58.166203 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b0271-f647-4c63-9d55-162b098b6709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:58.166894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0271-f647-4c63-9d55-162b098b6709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:58.167176 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0271-f647-4c63-9d55-162b098b6709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:58.167456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0271-f647-4c63-9d55-162b098b6709 admin admin] PATH_INFO: `/v3/projects/a2832893d5ff4e66b94339c67e2370f3/users/a333e1dbf81c44c698b6af949ae2c5cb/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:58.169998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b0271-f647-4c63-9d55-162b098b6709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2832893d5ff4e66b94339c67e2370f3, user_id=a333e1dbf81c44c698b6af949ae2c5cb, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:58.173028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b0271-f647-4c63-9d55-162b098b6709 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:58.176167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:58.189537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1387/2810] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:58 2022] PUT /identity/v3/projects/a2832893d5ff4e66b94339c67e2370f3/users/a333e1dbf81c44c698b6af949ae2c5cb/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:58.203276 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:58.203578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:58.203929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:58.229261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca24e-9c88-475e-b3f2-c2442cc4bcac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:58.245585 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] MFA Rules not processed for user `a333e1dbf81c44c698b6af949ae2c5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:58.247146 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:58.268266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:58.296503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:58.297031 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:58.297315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:58.297842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:58.360459 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1388/2811] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:19:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:58.362480 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:58.365059 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:58.394935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:58.401856 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:58.423445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1424/2812] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:58.437959 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:58.445287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:58.480939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40178cd-3cd3-4b14-917a-7a853607dbee service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:58.494294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:58.495898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:58.496357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:58.496799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:58.502535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:58.505696 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850d51b-49d2-497a-bf26-a7a7568599f7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:58.528437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:58.528437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:58.528437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:58.528437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:58.633568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:58.636658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:58.644913 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:58.649767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:58.656048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1425/2813] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:58.669251 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1389/2814] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:19:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:20:00.847938 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:00.861874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b531ce-d523-4a54-ab4a-b2aab08e25ca service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:00.869144 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:00.870704 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:00.871320 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:00.872827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:00.958223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:00.994182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:00.997268 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:01.015091 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1426/2815] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:03.375495 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:03.396672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcbea0-5a27-4582-bdb2-e839b6169785 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:03.410642 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:03.411557 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:03.411955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:03.412411 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:03.431688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:03.431688 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:03.467064 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1390/2816] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:08.399874 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:08.456852 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:08.514693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69251b82-ddf9-4d10-8a9f-d2faea0a2260 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:08.523645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:08.523965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:08.524183 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:08.524486 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] PATH_INFO: `/v3/users/30dd191837464d258d563eac37683da5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:08.526629 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30dd191837464d258d563eac37683da5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:08.530094 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:08.569212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:08.628613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36667334-8dca-4b99-83ab-024419411d8a service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:08.637754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:08.639545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:08.639793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:08.640494 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] PATH_INFO: `/v3/users/8b1fcf89e61b4f3fadb3af28310bc529` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:08.643428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1fcf89e61b4f3fadb3af28310bc529)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:08.646294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30dd191837464d258d563eac37683da5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.646922 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:08.690608 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30dd191837464d258d563eac37683da5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.709526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30dd191837464d258d563eac37683da5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.711315 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30dd191837464d258d563eac37683da5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.734792 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7d33be45-558e-463b-8946-35f1991dd265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30dd191837464d258d563eac37683da5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.751084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1427/2817] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:20:08 2022] DELETE /identity/v3/users/30dd191837464d258d563eac37683da5 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:08.754524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:08.755892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:08.756389 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:08.792568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1fcf89e61b4f3fadb3af28310bc529'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.795245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:08.797086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:08.810052 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1fcf89e61b4f3fadb3af28310bc529'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.829168 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1fcf89e61b4f3fadb3af28310bc529'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.830145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1fcf89e61b4f3fadb3af28310bc529'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.841477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-17309587-bf8a-47bd-9029-0937984930ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1fcf89e61b4f3fadb3af28310bc529'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:08.858848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1391/2818] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:20:08 2022] DELETE /identity/v3/users/8b1fcf89e61b4f3fadb3af28310bc529 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:08.859717 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508f79-94d8-4af9-b37c-c73e4bebd63c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:08.860428 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508f79-94d8-4af9-b37c-c73e4bebd63c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:08.861040 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508f79-94d8-4af9-b37c-c73e4bebd63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:08.906327 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7e508f79-94d8-4af9-b37c-c73e4bebd63c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:08.908456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7e508f79-94d8-4af9-b37c-c73e4bebd63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:08.928085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:08.966105 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1428/2819] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:08.991036 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:08.993184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:09.126625 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1392/2820] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:09.134289 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d395e1-12f6-49af-96e4-24bd76d80af1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:09.137379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508f79-94d8-4af9-b37c-c73e4bebd63c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:09.226040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:09.227422 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:09.228681 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:09.229366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:09.256466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508f79-94d8-4af9-b37c-c73e4bebd63c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:09.256926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:09.259937 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:09.260961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:09.261708 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:09.262094 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:09.262471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:09.278882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:09.281730 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:09.294737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1429/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:09.308108 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1393/2822] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:20:10.099945 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:10.101784 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:10.137626 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e355bf-5038-40bd-8c40-2c4cb1dc734b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:10.157457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:10.158448 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.158898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.159304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] PATH_INFO: `/v3/projects/d1a9b75e7c8a463a9d0c24483fbd6e0c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.167517 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a9b75e7c8a463a9d0c24483fbd6e0c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:10.170359 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:10.293403 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:10.295810 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:10.314564 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:10.335618 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e6af9-4a73-40aa-8327-57b02f857fbe service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:10.336787 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a9b75e7c8a463a9d0c24483fbd6e0c', 'request_id': 'req-a1d34086-7413-4d36-adbf-30b6473e365a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.354877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a9b75e7c8a463a9d0c24483fbd6e0c', 'request_id': 'req-a1d34086-7413-4d36-adbf-30b6473e365a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.369503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a9b75e7c8a463a9d0c24483fbd6e0c', 'request_id': 'req-a1d34086-7413-4d36-adbf-30b6473e365a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.425541 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] The token cache is being invalidate because project d1a9b75e7c8a463a9d0c24483fbd6e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:20:10.426632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.428275 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a1d34086-7413-4d36-adbf-30b6473e365a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.432789 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1430/2823] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:20:10 2022] DELETE /identity/v3/projects/d1a9b75e7c8a463a9d0c24483fbd6e0c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:10.439401 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd9e3-4b91-41ba-b8dd-e7bf1a69b0d0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.441801 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd9e3-4b91-41ba-b8dd-e7bf1a69b0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.442515 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd9e3-4b91-41ba-b8dd-e7bf1a69b0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.475823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:10.478118 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.478739 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.479250 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] PATH_INFO: `/v3/projects/0865a2a25a5f4901b689e5c660a57ba6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.481889 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0865a2a25a5f4901b689e5c660a57ba6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:10.482256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3a4fd9e3-4b91-41ba-b8dd-e7bf1a69b0d0 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:10.485546 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3a4fd9e3-4b91-41ba-b8dd-e7bf1a69b0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:10.486028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:10.504231 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1431/2824] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:10.513635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9938325-9b21-4156-a464-945709a5785a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.518423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9938325-9b21-4156-a464-945709a5785a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.519014 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9938325-9b21-4156-a464-945709a5785a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.553512 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d9938325-9b21-4156-a464-945709a5785a None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:10.555790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d9938325-9b21-4156-a464-945709a5785a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:10.579770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1432/2825] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:10.586901 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9938325-9b21-4156-a464-945709a5785a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:10.609017 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9938325-9b21-4156-a464-945709a5785a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:10.616986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0865a2a25a5f4901b689e5c660a57ba6', 'request_id': 'req-1ec99f64-123c-471f-b610-4c03573f2cfc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.628116 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0865a2a25a5f4901b689e5c660a57ba6', 'request_id': 'req-1ec99f64-123c-471f-b610-4c03573f2cfc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.640259 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0865a2a25a5f4901b689e5c660a57ba6', 'request_id': 'req-1ec99f64-123c-471f-b610-4c03573f2cfc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.669420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] The token cache is being invalidate because project 0865a2a25a5f4901b689e5c660a57ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:20:10.670358 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.670496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:10.672313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.674619 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1394/2826] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:20:10 2022] DELETE /identity/v3/projects/0865a2a25a5f4901b689e5c660a57ba6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:10.680867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:10.683895 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:10.727837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:10.730008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.730008 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.730294 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.732677 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:20:10.732846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:10.735092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:10.756502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1433/2827] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:20:10.762203 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:10.764971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:10.782589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57af12-6ee8-42d4-a2b9-c75b56aca8c2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:10.788186 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:10.788868 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.789147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.789420 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.796855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:10.796855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:10.800691 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99f64-123c-471f-b610-4c03573f2cfc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:10.821322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:10.821322 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.822106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.822106 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] PATH_INFO: `/v3/users/8757a267268c48d0bbc266b10377b89a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.823737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8757a267268c48d0bbc266b10377b89a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:10.827002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:10.841703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1434/2828] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:20:10 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:20:10.847922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:10.908120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:10.925586 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8757a267268c48d0bbc266b10377b89a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.942968 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8757a267268c48d0bbc266b10377b89a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.946843 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d390a-0fbe-4363-b312-341bae9dd891 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:10.953939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:10.954693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.954955 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.956258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.956258 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:10.957819 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:10.960241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8757a267268c48d0bbc266b10377b89a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.960986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8757a267268c48d0bbc266b10377b89a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.972512 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7cc2b665-0949-4350-95c3-d8ea7e13d557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8757a267268c48d0bbc266b10377b89a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:10.983747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1395/2829] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:20:10 2022] DELETE /identity/v3/users/8757a267268c48d0bbc266b10377b89a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:10.986085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4093cc-d555-4086-b4e6-d7bcd7101c84 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:10.986568 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4093cc-d555-4086-b4e6-d7bcd7101c84 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:10.986882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4093cc-d555-4086-b4e6-d7bcd7101c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:10.991137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1435/2830] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:20:10 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:20:10.993358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:10.995358 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:11.011345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ae4093cc-d555-4086-b4e6-d7bcd7101c84 None None] MFA Rules not processed for user `3c5f2dea7bab4b83b09c0c61b21e9700`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:11.012713 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ae4093cc-d555-4086-b4e6-d7bcd7101c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:11.020503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ceacd-1678-4bee-a2b6-b07bfe2a4b75 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:11.105699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1396/2831] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:11.133219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:11.134344 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:11.134835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:11.135331 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:11.136583 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:11.139894 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:11.162039 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1436/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:11.163874 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4093cc-d555-4086-b4e6-d7bcd7101c84 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:11.176091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4093cc-d555-4086-b4e6-d7bcd7101c84 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:11.189011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921c0d3-a811-4de7-9318-da334f5e4802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:11.190258 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921c0d3-a811-4de7-9318-da334f5e4802 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:11.191366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921c0d3-a811-4de7-9318-da334f5e4802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:11.191366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921c0d3-a811-4de7-9318-da334f5e4802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:11.191892 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921c0d3-a811-4de7-9318-da334f5e4802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:11.196207 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921c0d3-a811-4de7-9318-da334f5e4802 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:11.210747 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1397/2833] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:11.217241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:11.237049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3581ce-161e-4d16-87aa-fe911ef8af9a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:11.246632 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:11.247433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:11.247871 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:11.248360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] PATH_INFO: `/v3/projects/01cb2f734bce40f395387dcf5775031f/users/6c3cf41097514ad29c0975651a4d4615/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:11.271627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01cb2f734bce40f395387dcf5775031f, user_id=6c3cf41097514ad29c0975651a4d4615, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:11.275830 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:11.298640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1437/2834] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:20:11 2022] PUT /identity/v3/projects/01cb2f734bce40f395387dcf5775031f/users/6c3cf41097514ad29c0975651a4d4615/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:11.314584 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:11.315463 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:11.315754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:11.353914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] MFA Rules not processed for user `6c3cf41097514ad29c0975651a4d4615`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:11.356363 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:11.484850 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1398/2835] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:20:11.509423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:11.512818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:11.531152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:11.668671 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5094-e5e8-4821-b4ba-31feef94e0c1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:11.675313 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aba60973c5e4aa7b0b680acb24d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:11.677735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:11.677735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:11.677735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:11.683277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:11.685833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:11.695284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:11.697311 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:11.712243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1438/2836] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:11.763365 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:11.822101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60847be5-abce-41e8-bb8d-b43469ca66b3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:11.830046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:11.830976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:11.831260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:11.831653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] PATH_INFO: `/v3/projects/b57fc8e37bc44dc781534fe90d73cb67` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:11.832925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57fc8e37bc44dc781534fe90d73cb67)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:11.835961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:11.925993 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57fc8e37bc44dc781534fe90d73cb67', 'request_id': 'req-7edf0899-7144-4188-9baa-0d46c8be94e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:11.938694 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57fc8e37bc44dc781534fe90d73cb67', 'request_id': 'req-7edf0899-7144-4188-9baa-0d46c8be94e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:11.956941 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57fc8e37bc44dc781534fe90d73cb67', 'request_id': 'req-7edf0899-7144-4188-9baa-0d46c8be94e6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:11.978456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] The token cache is being invalidate because project b57fc8e37bc44dc781534fe90d73cb67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:20:11.979143 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:11.980142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:11.982093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1399/2837] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:20:11 2022] DELETE /identity/v3/projects/b57fc8e37bc44dc781534fe90d73cb67 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:11.986568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:12.002199 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec806ad-ac5f-4118-9e74-1b6940ace89e service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:12.105178 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:12.105996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:12.106281 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:12.106788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] PATH_INFO: `/v3/users/a333e1dbf81c44c698b6af949ae2c5cb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:12.108637 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a333e1dbf81c44c698b6af949ae2c5cb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:12.112197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:12.191971 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a333e1dbf81c44c698b6af949ae2c5cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:12.215717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a333e1dbf81c44c698b6af949ae2c5cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:12.232851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a333e1dbf81c44c698b6af949ae2c5cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:12.234353 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a333e1dbf81c44c698b6af949ae2c5cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:12.250625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-79df55e9-2e23-4fae-b498-f4a5ef105082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a333e1dbf81c44c698b6af949ae2c5cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:12.278146 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1439/2838] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:20:11 2022] DELETE /identity/v3/users/a333e1dbf81c44c698b6af949ae2c5cb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:20:12.756011 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:12.758493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:12.791379 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0899-7144-4188-9baa-0d46c8be94e6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:12.911164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:12.912114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:12.912581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:12.912992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] PATH_INFO: `/v3/projects/a2832893d5ff4e66b94339c67e2370f3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:12.914558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2832893d5ff4e66b94339c67e2370f3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:12.917782 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:12.998036 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2832893d5ff4e66b94339c67e2370f3', 'request_id': 'req-143dd03d-2893-4d0a-9266-d5a28ff16032'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:13.009496 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2832893d5ff4e66b94339c67e2370f3', 'request_id': 'req-143dd03d-2893-4d0a-9266-d5a28ff16032'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:13.022318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2832893d5ff4e66b94339c67e2370f3', 'request_id': 'req-143dd03d-2893-4d0a-9266-d5a28ff16032'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:13.032847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:13.057355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] The token cache is being invalidate because project a2832893d5ff4e66b94339c67e2370f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:20:13.058558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:13.060740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-143dd03d-2893-4d0a-9266-d5a28ff16032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:13.070615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1400/2839] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:20:12 2022] DELETE /identity/v3/projects/a2832893d5ff4e66b94339c67e2370f3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:13.072649 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90707-5434-4adb-a296-e59a0783f3c1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.073194 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90707-5434-4adb-a296-e59a0783f3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.073566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90707-5434-4adb-a296-e59a0783f3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.115234 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ffd90707-5434-4adb-a296-e59a0783f3c1 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:13.118066 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ffd90707-5434-4adb-a296-e59a0783f3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:13.154131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:13.229081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1440/2840] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:13.247671 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177635d-6dc2-4d3f-922d-6b7929b772d8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.248299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177635d-6dc2-4d3f-922d-6b7929b772d8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.248783 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177635d-6dc2-4d3f-922d-6b7929b772d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.294280 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6177635d-6dc2-4d3f-922d-6b7929b772d8 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:13.296289 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6177635d-6dc2-4d3f-922d-6b7929b772d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:13.308737 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1401/2841] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:13.318943 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90707-5434-4adb-a296-e59a0783f3c1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:13.342593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90707-5434-4adb-a296-e59a0783f3c1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:13.353325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:13.354325 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.355061 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.355477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.356913 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:20:13.357699 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:13.359999 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:13.387743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1441/2842] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:20:13.395318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177635d-6dc2-4d3f-922d-6b7929b772d8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:13.414944 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177635d-6dc2-4d3f-922d-6b7929b772d8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:13.426387 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:13.427476 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.428028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.428528 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.429847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:13.433115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:13.487703 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1402/2843] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:20:13 2022] POST /identity/v3/projects => generated 389 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:20:13.494238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:13.513513 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e83f36-b410-47e4-8789-586960ef1b60 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:13.646554 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:13.647740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.648200 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.648598 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.649503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:13.651970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:13.703951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1442/2844] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:20:13 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:20:13.709343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:13.727442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bd2ba-5601-42a3-b6bb-fa92a9b534fd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:13.735985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:13.737083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.738149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.738149 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.740355 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:13.742162 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:13.763637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1403/2845] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:13.770641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:13.787770 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735e30b-a69d-419c-b2de-8341f450ca1e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:13.795753 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16a5c9-dc0d-402c-b6e8-2c864062efd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:13.796573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a5c9-dc0d-402c-b6e8-2c864062efd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.797090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a5c9-dc0d-402c-b6e8-2c864062efd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.797521 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16a5c9-dc0d-402c-b6e8-2c864062efd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.798675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16a5c9-dc0d-402c-b6e8-2c864062efd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:13.801159 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16a5c9-dc0d-402c-b6e8-2c864062efd1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:13.823157 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1443/2846] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:13.828873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:13.843384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882932b1-01ec-4ffe-b7b7-9f6a6167223f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:13.859975 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:13.860998 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.861413 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.861861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] PATH_INFO: `/v3/projects/10a25433802a4582b5545c9c96f8e1ed/users/75d7b66c41fe4bd5995ecbd31a26ed92/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.886681 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a25433802a4582b5545c9c96f8e1ed, user_id=75d7b66c41fe4bd5995ecbd31a26ed92, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:13.891616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:13.911827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1404/2847] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:20:13 2022] PUT /identity/v3/projects/10a25433802a4582b5545c9c96f8e1ed/users/75d7b66c41fe4bd5995ecbd31a26ed92/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:20:13.928768 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2ff47-59a3-417e-97e1-e3d8acd1be4b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:13.929228 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2ff47-59a3-417e-97e1-e3d8acd1be4b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:13.929740 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2ff47-59a3-417e-97e1-e3d8acd1be4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:13.970006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cdb2ff47-59a3-417e-97e1-e3d8acd1be4b None None] MFA Rules not processed for user `75d7b66c41fe4bd5995ecbd31a26ed92`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:13.971317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cdb2ff47-59a3-417e-97e1-e3d8acd1be4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:14.106725 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1444/2848] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:14.125074 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:14.128825 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:14.246710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af405586-b313-4cb5-bc00-fae725d7ec67 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:14.260971 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:14.262329 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:14.265415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:14.265415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:14.295767 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:14.299121 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:14.322593 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1405/2849] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:14.780238 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:14.781174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:14.781380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:14.832299 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:14.834371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:14.856909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:14.963119 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1445/2850] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 13:20:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:14.995204 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:14.998205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:15.179141 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dfd90-1e53-44fc-97a1-7f2896ef734f service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:15.189830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:15.192796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:15.192796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:15.192796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:15.205294 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:15.207004 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:15.222382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1406/2851] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:20:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:20:24.943377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=16 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:25.213986 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:31.188569 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:31.190619 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:31.346332 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed26d-743c-4f81-8430-20f58834e9dd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:31.361882 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce290e8e-d20c-444f-8fcc-f7aa8e2ab435 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:31.362684 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290e8e-d20c-444f-8fcc-f7aa8e2ab435 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:31.363006 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290e8e-d20c-444f-8fcc-f7aa8e2ab435 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:31.363388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290e8e-d20c-444f-8fcc-f7aa8e2ab435 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:31.365552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ce290e8e-d20c-444f-8fcc-f7aa8e2ab435 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:31.388122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce290e8e-d20c-444f-8fcc-f7aa8e2ab435 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:31.393710 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce290e8e-d20c-444f-8fcc-f7aa8e2ab435 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:31.421011 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1446/2852] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:20:41.413994 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=34 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:50.870915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:51.013366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f69b-9148-4bf9-804a-73334ad6f2ad service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:51.018213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:51.018876 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:51.019155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:51.019432 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] PATH_INFO: `/v3/users/75d7b66c41fe4bd5995ecbd31a26ed92` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:51.022068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d7b66c41fe4bd5995ecbd31a26ed92)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:51.024006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:51.095422 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d7b66c41fe4bd5995ecbd31a26ed92'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.110809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d7b66c41fe4bd5995ecbd31a26ed92'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.137946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d7b66c41fe4bd5995ecbd31a26ed92'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.139325 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d7b66c41fe4bd5995ecbd31a26ed92'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.150029 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d7b66c41fe4bd5995ecbd31a26ed92'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.160065 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1407/2853] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:20:50 2022] DELETE /identity/v3/users/75d7b66c41fe4bd5995ecbd31a26ed92 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:51.166539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:51.166539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:51.166539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:51.192465 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:51.194161 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:51.280458 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1447/2854] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:51.299078 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:51.300578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:51.375530 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00754e59-c466-44c8-8ecb-48817adc79c0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:51.381120 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac43ed4-2392-4e76-9fc1-a1e0a36163e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:51.382041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac43ed4-2392-4e76-9fc1-a1e0a36163e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:51.382429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac43ed4-2392-4e76-9fc1-a1e0a36163e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:51.382793 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac43ed4-2392-4e76-9fc1-a1e0a36163e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:51.397234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac43ed4-2392-4e76-9fc1-a1e0a36163e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:51.398778 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac43ed4-2392-4e76-9fc1-a1e0a36163e0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:51.410114 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1408/2855] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:51.744475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:51.746808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:51.763834 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0b48e-6d4b-43bf-a710-b06d716f3a5c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:51.772575 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:51.773303 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:51.773889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:51.774354 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] PATH_INFO: `/v3/projects/10a25433802a4582b5545c9c96f8e1ed` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:51.775790 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a25433802a4582b5545c9c96f8e1ed)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:51.778495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:51.838606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a25433802a4582b5545c9c96f8e1ed', 'request_id': 'req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.849578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a25433802a4582b5545c9c96f8e1ed', 'request_id': 'req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.861535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a25433802a4582b5545c9c96f8e1ed', 'request_id': 'req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.877223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] The token cache is being invalidate because project 10a25433802a4582b5545c9c96f8e1ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:20:51.877524 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.878329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fedf5ee-7a3c-4b9d-8973-c4cb5f08f4a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:51.879861 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1448/2856] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:20:51 2022] DELETE /identity/v3/projects/10a25433802a4582b5545c9c96f8e1ed => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:51.885415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3ba60-fe44-4589-9995-2d903ca4229f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:51.886105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3ba60-fe44-4589-9995-2d903ca4229f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:51.886105 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3ba60-fe44-4589-9995-2d903ca4229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:51.909915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dca3ba60-fe44-4589-9995-2d903ca4229f None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:51.915461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dca3ba60-fe44-4589-9995-2d903ca4229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:51.936375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:51.990972 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1409/2857] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:52.002616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3b50-6ee2-4801-8f6d-535152bddd54 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.003004 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3b50-6ee2-4801-8f6d-535152bddd54 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.003430 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3b50-6ee2-4801-8f6d-535152bddd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.028914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9cfb3b50-6ee2-4801-8f6d-535152bddd54 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:52.030243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9cfb3b50-6ee2-4801-8f6d-535152bddd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:52.047227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1449/2858] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:52.053308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3ba60-fe44-4589-9995-2d903ca4229f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:52.064915 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3ba60-fe44-4589-9995-2d903ca4229f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:52.071917 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:52.072556 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.072937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.073153 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.073908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:20:52.074270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:52.076676 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:52.094563 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1410/2859] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:20:52.099979 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb3b50-6ee2-4801-8f6d-535152bddd54 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:52.112896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb3b50-6ee2-4801-8f6d-535152bddd54 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:52.118915 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:52.119545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.119878 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.120478 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.121016 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:52.123261 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:52.178702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1450/2860] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:20:52 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:20:52.180803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:52.196415 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5fd53-fe7d-44ba-9634-7f7eeeab9fd1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:52.282965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:52.283592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.283893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.284165 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.285623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:52.286853 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:52.322772 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1411/2861] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:20:52 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:20:52.326566 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:52.338506 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16432c-bab5-41ac-ba0b-6f2f44ea68f0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:52.345947 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:52.346582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.346889 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.347193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.347827 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:52.349846 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:52.363182 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1451/2862] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:52.369087 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:52.380847 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415661b-120c-4728-a5f3-809bf02363af admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:52.390692 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c926ef-af93-4d28-b31a-f7132b1c4cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:52.391092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c926ef-af93-4d28-b31a-f7132b1c4cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.391369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c926ef-af93-4d28-b31a-f7132b1c4cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.391684 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c926ef-af93-4d28-b31a-f7132b1c4cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.392292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c926ef-af93-4d28-b31a-f7132b1c4cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:52.394427 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c926ef-af93-4d28-b31a-f7132b1c4cbb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:52.412021 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1412/2863] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:52.413698 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:52.427589 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdeb1-5d52-450d-a760-6cbf93b28639 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:52.441749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:52.442950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.443472 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.443957 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] PATH_INFO: `/v3/projects/695a571420de439581eb614b16b327aa/users/43a6a0b75e48405ca6385ea62768d4d9/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.468527 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695a571420de439581eb614b16b327aa, user_id=43a6a0b75e48405ca6385ea62768d4d9, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:52.471423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:52.491578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1452/2864] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:20:52 2022] PUT /identity/v3/projects/695a571420de439581eb614b16b327aa/users/43a6a0b75e48405ca6385ea62768d4d9/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:52.496169 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88375e3-87c4-4ac6-80d7-29b3a0f94f70 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.496481 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88375e3-87c4-4ac6-80d7-29b3a0f94f70 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.496724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88375e3-87c4-4ac6-80d7-29b3a0f94f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.521809 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b88375e3-87c4-4ac6-80d7-29b3a0f94f70 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:52.523091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b88375e3-87c4-4ac6-80d7-29b3a0f94f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:52.625578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1413/2865] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:52.640616 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:52.644902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:52.679074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:52.725864 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efade69-a4fe-40ef-8630-086513c59f80 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:52.732545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:52.733620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:52.734095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:52.734580 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:52.759148 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:52.761717 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:52.773177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1453/2866] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:58.324843 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3161f0-68e9-410c-af6c-ed6f7673931b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:58.325849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3161f0-68e9-410c-af6c-ed6f7673931b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:58.326616 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3161f0-68e9-410c-af6c-ed6f7673931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:58.373610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9e3161f0-68e9-410c-af6c-ed6f7673931b None None] MFA Rules not processed for user `43a6a0b75e48405ca6385ea62768d4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:58.375797 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9e3161f0-68e9-410c-af6c-ed6f7673931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:58.416634 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:58.536274 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1414/2867] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:20:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:58.550578 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:58.562507 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a311f-c785-41c9-9f6d-d2135f2c2b6c service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:58.574877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:58.575970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:58.576284 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:58.576744 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:58.578097 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:58.581548 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:58.625445 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1454/2868] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:20:58 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:20:58.627308 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3161f0-68e9-410c-af6c-ed6f7673931b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:58.644293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3161f0-68e9-410c-af6c-ed6f7673931b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:58.742643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:58.743845 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:58.744447 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:58.745157 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:58.746343 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:58.749297 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:58.801712 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1415/2869] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:20:58 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:20:58.806760 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:58.822423 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e12e-deef-48bc-8bf6-e6fccffe3553 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:58.827568 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:58.828188 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:58.828645 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:58.828922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:58.829618 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:58.831286 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:58.844510 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1455/2870] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:20:58.848811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:58.861388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821ca4f-9ad5-4f7f-a237-ccdf108bfc9a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:58.866456 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:58.867134 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:58.867408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:58.867680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] PATH_INFO: `/v3/projects/4ea43de593384c98848120f29664c33b/users/0ec3f41c43df4b16b8cfcf7a026cba20/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:58.892803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea43de593384c98848120f29664c33b, user_id=0ec3f41c43df4b16b8cfcf7a026cba20, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:58.898199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:58.922472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1416/2871] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:20:58 2022] PUT /identity/v3/projects/4ea43de593384c98848120f29664c33b/users/0ec3f41c43df4b16b8cfcf7a026cba20/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:20:58.926835 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:58.940310 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc516608-e67c-4cad-b010-e13861b3c982 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:59.025283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:59.026545 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:59.027085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:59.027557 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:59.028743 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:59.032098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:59.056762 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1456/2872] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:59.063544 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:59.078104 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16f68-fd3b-49c2-8ffc-1ce41dab0313 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:59.084354 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab743889-1bbc-4774-a41d-37f198bfbd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:59.085193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab743889-1bbc-4774-a41d-37f198bfbd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:59.085510 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab743889-1bbc-4774-a41d-37f198bfbd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:59.086159 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab743889-1bbc-4774-a41d-37f198bfbd4d admin admin] PATH_INFO: `/v3/projects/4ea43de593384c98848120f29664c33b/users/0ec3f41c43df4b16b8cfcf7a026cba20/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:59.089547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab743889-1bbc-4774-a41d-37f198bfbd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea43de593384c98848120f29664c33b, user_id=0ec3f41c43df4b16b8cfcf7a026cba20, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:59.091926 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab743889-1bbc-4774-a41d-37f198bfbd4d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:59.105440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1417/2873] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:20:59 2022] PUT /identity/v3/projects/4ea43de593384c98848120f29664c33b/users/0ec3f41c43df4b16b8cfcf7a026cba20/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:03.385903 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:03.388166 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:03.409611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb71f73-c6d1-40f1-8c3f-afb9b1c972b4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:03.599176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:03.601406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:03.602377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:03.603241 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] PATH_INFO: `/v3/users/6c3cf41097514ad29c0975651a4d4615` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:03.606056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c3cf41097514ad29c0975651a4d4615)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:03.612886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:03.714439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:03.796042 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c3cf41097514ad29c0975651a4d4615'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:03.839828 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c3cf41097514ad29c0975651a4d4615'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:03.854553 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c3cf41097514ad29c0975651a4d4615'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:03.855470 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c3cf41097514ad29c0975651a4d4615'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:03.877533 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c3cf41097514ad29c0975651a4d4615'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:03.888053 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1457/2874] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:21:03 2022] DELETE /identity/v3/users/6c3cf41097514ad29c0975651a4d4615 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:03.894983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:03.894983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:03.895820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:03.939824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:03.941561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:04.069815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1418/2875] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:04.082886 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:04.085818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:04.230928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04151ce2-93b8-47b7-a377-79e92d2799c6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:04.237328 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee6f41-29ce-488d-9222-9db269fd2e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:04.246694 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee6f41-29ce-488d-9222-9db269fd2e0f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:04.247158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee6f41-29ce-488d-9222-9db269fd2e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:04.247697 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee6f41-29ce-488d-9222-9db269fd2e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:04.270082 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee6f41-29ce-488d-9222-9db269fd2e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:04.271559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee6f41-29ce-488d-9222-9db269fd2e0f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:04.324576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1458/2876] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:05.069442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:05.072886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:05.097142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db48a-8384-4f02-a935-e90d337242bd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:05.111093 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:05.112382 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:05.112912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:05.114002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] PATH_INFO: `/v3/projects/01cb2f734bce40f395387dcf5775031f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:05.116966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01cb2f734bce40f395387dcf5775031f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:05.119867 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:05.141191 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:05.229231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01cb2f734bce40f395387dcf5775031f', 'request_id': 'req-53f7236d-d2e8-40e1-aeea-5714104c19a0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:05.244373 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01cb2f734bce40f395387dcf5775031f', 'request_id': 'req-53f7236d-d2e8-40e1-aeea-5714104c19a0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:05.257401 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01cb2f734bce40f395387dcf5775031f', 'request_id': 'req-53f7236d-d2e8-40e1-aeea-5714104c19a0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:05.294166 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] The token cache is being invalidate because project 01cb2f734bce40f395387dcf5775031f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:21:05.294800 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:05.295695 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-53f7236d-d2e8-40e1-aeea-5714104c19a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:05.299384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1419/2877] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:21:05 2022] DELETE /identity/v3/projects/01cb2f734bce40f395387dcf5775031f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:05.304700 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9b5e-0249-4fc6-a4b2-39c5d3bbad14 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:05.305852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9b5e-0249-4fc6-a4b2-39c5d3bbad14 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:05.306492 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9b5e-0249-4fc6-a4b2-39c5d3bbad14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:05.368638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-668f9b5e-0249-4fc6-a4b2-39c5d3bbad14 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:05.371069 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-668f9b5e-0249-4fc6-a4b2-39c5d3bbad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:05.558287 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1459/2878] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:05.565849 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9757eed-2771-4753-8f5d-983937351a16 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:05.566300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9757eed-2771-4753-8f5d-983937351a16 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:05.566960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9757eed-2771-4753-8f5d-983937351a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:05.600254 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e9757eed-2771-4753-8f5d-983937351a16 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:05.601323 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e9757eed-2771-4753-8f5d-983937351a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:05.613552 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1420/2879] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:05.618896 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f9b5e-0249-4fc6-a4b2-39c5d3bbad14 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:05.636847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f9b5e-0249-4fc6-a4b2-39c5d3bbad14 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:05.646361 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:05.647404 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:05.647899 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:05.648377 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:05.650043 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:21:05.650676 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:05.655109 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:05.685018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1460/2880] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:21:05.692466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9757eed-2771-4753-8f5d-983937351a16 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:05.704613 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9757eed-2771-4753-8f5d-983937351a16 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:05.719286 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:05.742824 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:05.744489 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:05.757771 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:05.759406 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:05.771092 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:05.834375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1421/2881] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:21:05 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:21:05.837340 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:05.872964 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faeee6-263a-47a0-b96c-83a7a9d4272e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:05.968221 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:06.069818 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:06.071400 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.071969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.072266 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.073123 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:06.075594 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:06.132071 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1461/2882] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:21:05 2022] POST /identity/v3/users => generated 288 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:21:06.132742 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:06.152565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c320d-f5bc-4c6c-b3ac-14b088953c30 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:06.175958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d65a3d-a1f4-4051-82dd-6df1440e8df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:06.176740 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d65a3d-a1f4-4051-82dd-6df1440e8df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.177026 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d65a3d-a1f4-4051-82dd-6df1440e8df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.177302 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d65a3d-a1f4-4051-82dd-6df1440e8df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.178213 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d65a3d-a1f4-4051-82dd-6df1440e8df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:06.179880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d65a3d-a1f4-4051-82dd-6df1440e8df7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:06.198081 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1422/2883] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:06.204621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:06.222656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.223262 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.223537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.239961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351e5c8-3d18-4da2-af80-0cb131da4fc8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:06.257599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:06.258319 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.258703 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.264570 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.265048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] MFA Rules not processed for user `0ec3f41c43df4b16b8cfcf7a026cba20`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:06.266079 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:06.267973 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:06.268293 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:06.289304 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1462/2884] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:06.296522 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:06.310245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a721bf6-ffca-444b-af99-8746b26ea1ac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:06.316752 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1644e-80f5-4a81-87ec-0b44c4406e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:06.318021 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1644e-80f5-4a81-87ec-0b44c4406e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.318516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1644e-80f5-4a81-87ec-0b44c4406e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.318996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1644e-80f5-4a81-87ec-0b44c4406e30 admin admin] PATH_INFO: `/v3/projects/7848147338344ccdaf44a380adcd7d83/users/5823f4d4b0ad408094ed90adf7599c8e/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.342909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1644e-80f5-4a81-87ec-0b44c4406e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7848147338344ccdaf44a380adcd7d83, user_id=5823f4d4b0ad408094ed90adf7599c8e, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:06.346296 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1644e-80f5-4a81-87ec-0b44c4406e30 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:06.370842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1463/2885] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:21:06 2022] PUT /identity/v3/projects/7848147338344ccdaf44a380adcd7d83/users/5823f4d4b0ad408094ed90adf7599c8e/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:06.395251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.395251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.395251 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.462762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1423/2886] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:21:06.476908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:06.479784 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:06.502807 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] MFA Rules not processed for user `5823f4d4b0ad408094ed90adf7599c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:06.504143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:06.621992 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:06.671180 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1464/2887] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:06.707090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:06.710551 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:06.724561 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe99378-20eb-4a52-8d0e-caa3ebed9fa7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:06.833519 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df59ff-04a1-410f-b9f3-f3ca90063b2d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:06.843809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:06.844860 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.846085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.846495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.851508 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:06.854374 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:06.859032 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:06.862378 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:06.864748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:06.873599 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:06.876464 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:06.906143 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1465/2888] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:06.980205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:06.983299 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:07.008334 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1424/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:07.839772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:07.839772 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:07.887177 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:07.966360 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94566b-abed-4adf-8f21-7ad0d3d7addd service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:07.983388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f703ef-3e62-4289-b8a5-04e9e95ee834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:07.984537 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f703ef-3e62-4289-b8a5-04e9e95ee834 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:07.984972 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f703ef-3e62-4289-b8a5-04e9e95ee834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:07.985438 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f703ef-3e62-4289-b8a5-04e9e95ee834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:07.992007 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e7f703ef-3e62-4289-b8a5-04e9e95ee834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:08.171607 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f703ef-3e62-4289-b8a5-04e9e95ee834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:08.184460 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f703ef-3e62-4289-b8a5-04e9e95ee834 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:08.212056 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1466/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:16.996292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=66 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:18.204573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=40 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:20.984896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:21.076015 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beba2d-d8ce-44b8-a2bb-463d7328ad4b service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:21.081218 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:21.081914 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:21.082194 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:21.082465 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] PATH_INFO: `/v3/users/5823f4d4b0ad408094ed90adf7599c8e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:21.083679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5823f4d4b0ad408094ed90adf7599c8e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:21.085637 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:21.145898 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5823f4d4b0ad408094ed90adf7599c8e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.156966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5823f4d4b0ad408094ed90adf7599c8e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.173961 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5823f4d4b0ad408094ed90adf7599c8e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.174880 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5823f4d4b0ad408094ed90adf7599c8e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.184270 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5823f4d4b0ad408094ed90adf7599c8e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.191755 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1425/2891] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:21:20 2022] DELETE /identity/v3/users/5823f4d4b0ad408094ed90adf7599c8e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:21.196799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:21.197393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:21.197393 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:21.223495 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:21.225640 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:21.324810 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1467/2892] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:21.336699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:21.338848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:21.414028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240f71d-e047-4b51-9df3-bfaad6e6cc0a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:21.420886 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006a841-6bd2-4417-b82f-16c926fa3520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:21.422008 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006a841-6bd2-4417-b82f-16c926fa3520 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:21.422395 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006a841-6bd2-4417-b82f-16c926fa3520 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:21.423241 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006a841-6bd2-4417-b82f-16c926fa3520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:21.435461 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006a841-6bd2-4417-b82f-16c926fa3520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:21.437282 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006a841-6bd2-4417-b82f-16c926fa3520 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:21.449811 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1426/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:21.826095 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:21.828336 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:21.848842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e1d00-c682-4d6f-9b0f-0c411c7480cd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:21.856879 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:21.857822 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:21.858167 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:21.858603 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] PATH_INFO: `/v3/projects/7848147338344ccdaf44a380adcd7d83` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:21.860615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7848147338344ccdaf44a380adcd7d83)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:21.864179 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:21.938108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7848147338344ccdaf44a380adcd7d83', 'request_id': 'req-f259b315-565b-44e6-a1ea-a79e08771008'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.948051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7848147338344ccdaf44a380adcd7d83', 'request_id': 'req-f259b315-565b-44e6-a1ea-a79e08771008'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.957713 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7848147338344ccdaf44a380adcd7d83', 'request_id': 'req-f259b315-565b-44e6-a1ea-a79e08771008'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.971939 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] The token cache is being invalidate because project 7848147338344ccdaf44a380adcd7d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:21:21.972317 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.972921 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f259b315-565b-44e6-a1ea-a79e08771008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:21.975424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1468/2894] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:21:21 2022] DELETE /identity/v3/projects/7848147338344ccdaf44a380adcd7d83 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:21.981776 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55166e-fe55-4995-a8f0-32fa6d0c9760 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:21.982266 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55166e-fe55-4995-a8f0-32fa6d0c9760 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:21.982785 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55166e-fe55-4995-a8f0-32fa6d0c9760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.006186 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1b55166e-fe55-4995-a8f0-32fa6d0c9760 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:22.008205 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1b55166e-fe55-4995-a8f0-32fa6d0c9760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:22.031743 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:22.092212 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1427/2895] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:22.101136 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af088903-ea13-4932-bee4-8db088f95893 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.101976 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af088903-ea13-4932-bee4-8db088f95893 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.102496 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af088903-ea13-4932-bee4-8db088f95893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.123850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-af088903-ea13-4932-bee4-8db088f95893 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:22.125074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-af088903-ea13-4932-bee4-8db088f95893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:22.133611 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1469/2896] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:21:22.139181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55166e-fe55-4995-a8f0-32fa6d0c9760 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:22.155437 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55166e-fe55-4995-a8f0-32fa6d0c9760 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:22.161988 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:22.162757 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.163041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.163316 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.164145 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:21:22.164500 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:22.166216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:22.179466 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1428/2897] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 13:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:21:22.187318 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af088903-ea13-4932-bee4-8db088f95893 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:22.195813 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af088903-ea13-4932-bee4-8db088f95893 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:22.201722 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:22.202384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.202625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.202890 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.203539 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:22.205516 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:22.241612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1470/2898] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:21:22 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:21:22.246937 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:22.263072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5cc5c-9dcf-4386-b3d7-b38844c41fe9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:22.343210 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:22.343833 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.344115 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.344388 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.345046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:22.347152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:22.378577 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1429/2899] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:21:22 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:21:22.383660 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:22.396142 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f670e7-5b2d-4b97-8633-7a4c6cecdaee admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:22.401544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:22.402197 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.402454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.402749 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.403394 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:22.405137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:22.416372 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1471/2900] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:22.420660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:22.431925 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f111-fa8c-48a0-9849-8715eeb9188c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:22.439155 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdb40a-c9e5-4101-85cf-8acb4ec3353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:22.440067 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdb40a-c9e5-4101-85cf-8acb4ec3353a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.440408 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdb40a-c9e5-4101-85cf-8acb4ec3353a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.440929 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdb40a-c9e5-4101-85cf-8acb4ec3353a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.441581 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdb40a-c9e5-4101-85cf-8acb4ec3353a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:22.444007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdb40a-c9e5-4101-85cf-8acb4ec3353a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:22.458216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1430/2901] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:22.463574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:22.475682 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50281e-2634-4a9c-a2a9-6e77988b09fd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:22.481638 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:22.482426 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.482711 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.482984 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] PATH_INFO: `/v3/projects/628aff1654fc4804bbbec0333d3537b2/users/abffc071dfac402087686ed80260f31d/roles/ebf589306f5f4cbd8bc523efc556cd3d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.499995 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=628aff1654fc4804bbbec0333d3537b2, user_id=abffc071dfac402087686ed80260f31d, role_id=ebf589306f5f4cbd8bc523efc556cd3d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:22.502851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:22.519321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1472/2902] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:21:22 2022] PUT /identity/v3/projects/628aff1654fc4804bbbec0333d3537b2/users/abffc071dfac402087686ed80260f31d/roles/ebf589306f5f4cbd8bc523efc556cd3d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:22.525601 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aa082-fd7a-4419-a86a-f441797e9c45 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.526176 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aa082-fd7a-4419-a86a-f441797e9c45 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.526702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aa082-fd7a-4419-a86a-f441797e9c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.552690 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9d5aa082-fd7a-4419-a86a-f441797e9c45 None None] MFA Rules not processed for user `ca1f33becd334f89a27a7404e04ed5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:22.554072 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9d5aa082-fd7a-4419-a86a-f441797e9c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:22.636912 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1431/2903] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:22.647081 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:22.649654 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:22.675018 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:22.734330 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86544719-a7aa-4676-b0bc-fc6813e1d4ca admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:22.738627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:22.739282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:22.739604 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:22.739904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:22.752047 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:22.753928 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:22.767715 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1473/2904] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:21:27.307436 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfde812-9997-4047-8a24-914adb5cf08d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:27.308084 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfde812-9997-4047-8a24-914adb5cf08d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:27.308520 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfde812-9997-4047-8a24-914adb5cf08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:27.346345 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ecfde812-9997-4047-8a24-914adb5cf08d None None] MFA Rules not processed for user `abffc071dfac402087686ed80260f31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:27.347565 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecfde812-9997-4047-8a24-914adb5cf08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:27.367956 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:27.434096 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1432/2905] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:27.438576 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:27.448850 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea6b6e-e531-4285-ab0b-e9b81e96477a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:27.454334 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:27.454970 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:27.455252 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:27.455693 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:27.456338 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:27.458357 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:27.490969 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1474/2906] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 13:21:27 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:21:27.495232 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfde812-9997-4047-8a24-914adb5cf08d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:27.508042 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfde812-9997-4047-8a24-914adb5cf08d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:27.582371 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:27.583002 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:27.584796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:27.584796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:27.584796 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:27.586440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:27.613174 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1433/2907] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 13:21:27 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:21:27.618806 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:27.631196 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68535c0d-4f7b-4c81-82d6-63284923fdf1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:27.637185 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:27.637922 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:27.639103 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:27.639466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:27.640195 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:27.641962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:27.661295 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1475/2908] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:27.667068 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:27.678304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531fc62-cc16-4fed-a044-bfb3887a0a0e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:27.683638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:27.684320 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:27.684595 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:27.684866 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] PATH_INFO: `/v3/projects/d9ce25b874db4eb1912b4d6def5e83f9/users/28b2936d3d34497b96a4dd4232afa56b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:27.704697 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ce25b874db4eb1912b4d6def5e83f9, user_id=28b2936d3d34497b96a4dd4232afa56b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:27.707531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:27.724516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1434/2909] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:21:27 2022] PUT /identity/v3/projects/d9ce25b874db4eb1912b4d6def5e83f9/users/28b2936d3d34497b96a4dd4232afa56b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:27.728853 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:27.739965 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909abfe-e1b3-4ec9-85d8-fa2217e25596 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:27.830530 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e623f8-8585-4df3-8381-e45041338f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:27.831424 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e623f8-8585-4df3-8381-e45041338f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:27.831898 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e623f8-8585-4df3-8381-e45041338f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:27.832454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e623f8-8585-4df3-8381-e45041338f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:27.833388 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e623f8-8585-4df3-8381-e45041338f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:27.835954 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e623f8-8585-4df3-8381-e45041338f6a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:27.853535 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1476/2910] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:27.858803 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:27.871231 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304067bd-94b9-4cc2-b668-33d48474168f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:27.876966 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fb041-8a60-4b57-97f5-531220705156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:27.878660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fb041-8a60-4b57-97f5-531220705156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:27.878660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fb041-8a60-4b57-97f5-531220705156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:27.878660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fb041-8a60-4b57-97f5-531220705156 admin admin] PATH_INFO: `/v3/projects/d9ce25b874db4eb1912b4d6def5e83f9/users/28b2936d3d34497b96a4dd4232afa56b/roles/2cff20de7350442a92ef150718ecf5c0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:27.880444 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fb041-8a60-4b57-97f5-531220705156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ce25b874db4eb1912b4d6def5e83f9, user_id=28b2936d3d34497b96a4dd4232afa56b, role_id=2cff20de7350442a92ef150718ecf5c0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:27.883445 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fb041-8a60-4b57-97f5-531220705156 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:27.896423 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1435/2911] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 13:21:27 2022] PUT /identity/v3/projects/d9ce25b874db4eb1912b4d6def5e83f9/users/28b2936d3d34497b96a4dd4232afa56b/roles/2cff20de7350442a92ef150718ecf5c0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:33.363634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:33.365041 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:33.365644 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:33.403737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] MFA Rules not processed for user `43a6a0b75e48405ca6385ea62768d4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:33.405132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:33.459000 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:33.459600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:33.459600 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:33.499466 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1477/2912] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:33.511769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] MFA Rules not processed for user `28b2936d3d34497b96a4dd4232afa56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:33.513464 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:33.572068 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:33.574223 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:33.599640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:33.617626 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1436/2913] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:33.623206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:33.640817 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:33.642829 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:33.666410 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2abf8a4-927e-41ba-981a-7e831aa30bc6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:33.674475 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:33.675380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:33.675669 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:33.675948 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:33.679586 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:33.773287 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee3621-3310-410b-b8a3-fd156cfb2290 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:33.783724 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:33.784396 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:33.784672 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:33.784946 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:33.786019 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:33.787761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:33.802049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1478/2914] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:21:33.907017 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:33.908364 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:33.925957 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1437/2915] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:34.439323 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:34.522005 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38709-b8b4-4ba8-a5e5-79b25ab32196 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:34.526870 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:34.527337 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:34.527624 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:34.527914 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:34.530571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:34.544573 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:34.546110 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:34.560391 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1479/2916] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:35.506842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:35.508931 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:35.519270 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:35.521951 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:35.534728 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeccb3-9f0f-4a9c-b58e-ec9b865b2c3e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:35.540247 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696e4a2-b416-4876-9ef2-1124c3b59b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:35.541158 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696e4a2-b416-4876-9ef2-1124c3b59b66 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:35.541433 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696e4a2-b416-4876-9ef2-1124c3b59b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:35.541741 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696e4a2-b416-4876-9ef2-1124c3b59b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:35.544685 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5696e4a2-b416-4876-9ef2-1124c3b59b66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:35.562100 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696e4a2-b416-4876-9ef2-1124c3b59b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:35.564476 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696e4a2-b416-4876-9ef2-1124c3b59b66 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:35.587931 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1480/2917] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:35.639516 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923e33-8039-4d8f-812b-83b6c805aaa5 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:35.645237 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:35.645908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:35.646429 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:35.646604 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:35.648903 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:35.669638 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:35.671631 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:35.684324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1438/2918] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:35.774207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec75fab-1b30-4942-9e76-9d19cfa448cb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:35.774633 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec75fab-1b30-4942-9e76-9d19cfa448cb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:35.775321 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec75fab-1b30-4942-9e76-9d19cfa448cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:35.816797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4ec75fab-1b30-4942-9e76-9d19cfa448cb None None] MFA Rules not processed for user `abffc071dfac402087686ed80260f31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:35.818152 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4ec75fab-1b30-4942-9e76-9d19cfa448cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:35.829981 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1481/2919] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:21:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:45.680056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:45.804620 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=75 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:51.660140 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:51.672315 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131cd92-9082-4c35-9c7a-307efe0518a8 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:51.676023 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:51.676620 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:51.676896 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:51.677545 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:51.679472 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:51.690753 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:51.692208 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:51.703844 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1439/2920] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:54.237873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec75fab-1b30-4942-9e76-9d19cfa448cb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:54.248865 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec75fab-1b30-4942-9e76-9d19cfa448cb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:54.253442 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:54.254098 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:54.254370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:54.254884 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:54.257256 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:54.328854 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:54.330343 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:54.341788 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1482/2921] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:22:01.697085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:04.336824 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:19.415548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:19.432815 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a571f-563d-4be9-a319-fdbc2291bfdd service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:19.439493 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be28f0-169a-4ee7-a2df-55b484e21d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:19.440129 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be28f0-169a-4ee7-a2df-55b484e21d53 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:19.440588 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be28f0-169a-4ee7-a2df-55b484e21d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:19.441242 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be28f0-169a-4ee7-a2df-55b484e21d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:19.456142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be28f0-169a-4ee7-a2df-55b484e21d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:19.458440 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be28f0-169a-4ee7-a2df-55b484e21d53 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:19.479872 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1440/2922] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:29.464260 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:50.943380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:50.958538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567bc14-49dd-4ea0-9578-3957dc2d4685 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:50.966872 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:50.968596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:50.968596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:50.969059 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:50.973368 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:50.991329 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:50.992950 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:51.011263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1483/2923] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:59.630016 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a31e9-dfc1-4f7a-9227-d74ccf375209 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:59.630653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a31e9-dfc1-4f7a-9227-d74ccf375209 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:59.630653 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a31e9-dfc1-4f7a-9227-d74ccf375209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:59.655049 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-857a31e9-dfc1-4f7a-9227-d74ccf375209 None None] MFA Rules not processed for user `1f9f3907a6d94b42b40343adca84bba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:59.656955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-857a31e9-dfc1-4f7a-9227-d74ccf375209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:59.733680 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1441/2924] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 13:23:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:59.776737 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:59.778625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:59.857085 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b5474-754f-4396-9c13-6cc2d49cd5cb service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:59.863064 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30f70cdf08d049668d16007e4ca44654', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:59.863686 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:59.863962 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:59.864230 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:59.874380 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:59.875997 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:59.885761 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1484/2925] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 13:23:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:24:03.601790 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a31e9-dfc1-4f7a-9227-d74ccf375209 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:03.620272 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a31e9-dfc1-4f7a-9227-d74ccf375209 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:03.626830 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:03.628537 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:03.629366 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:03.629883 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:03.631904 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:03.649375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:03.652007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:03.666487 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1442/2926] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:07.101120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:07.103058 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:07.180584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9a38e-3c30-403a-a0e5-27fd9afc9d4d service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:07.185406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:07.186137 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:07.187206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:07.187206 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] PATH_INFO: `/v3/users/3c5f2dea7bab4b83b09c0c61b21e9700` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:07.187837 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c5f2dea7bab4b83b09c0c61b21e9700)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:07.189823 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:07.250436 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c5f2dea7bab4b83b09c0c61b21e9700'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.273877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c5f2dea7bab4b83b09c0c61b21e9700'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.283326 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c5f2dea7bab4b83b09c0c61b21e9700'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.284131 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c5f2dea7bab4b83b09c0c61b21e9700'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.291582 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:07.293766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c5f2dea7bab4b83b09c0c61b21e9700'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.302692 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1485/2927] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:24:07 2022] DELETE /identity/v3/users/3c5f2dea7bab4b83b09c0c61b21e9700 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:24:07.518834 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:07.535405 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fcd87-e456-48c9-bcb8-ebb37faa9595 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:07.612469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:07.613139 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:07.613720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:07.613720 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] PATH_INFO: `/v3/projects/5f20e431b2b140ebb7c895ebd5954f67` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:07.614882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f20e431b2b140ebb7c895ebd5954f67)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:07.616823 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:07.676014 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f20e431b2b140ebb7c895ebd5954f67', 'request_id': 'req-937ec912-77e4-4095-b33b-c80fba15db0d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.682732 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f20e431b2b140ebb7c895ebd5954f67', 'request_id': 'req-937ec912-77e4-4095-b33b-c80fba15db0d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.690277 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f20e431b2b140ebb7c895ebd5954f67', 'request_id': 'req-937ec912-77e4-4095-b33b-c80fba15db0d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.705389 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] The token cache is being invalidate because project 5f20e431b2b140ebb7c895ebd5954f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:24:07.705908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.706346 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.708654 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1443/2928] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:24:07 2022] DELETE /identity/v3/projects/5f20e431b2b140ebb7c895ebd5954f67 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:24:07.714048 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:07.715574 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:07.730536 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e01f37-73d7-4e56-8629-9cae20df8187 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:07.805808 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:07.806453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:07.806731 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:07.807027 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] PATH_INFO: `/v3/users/2f45f33b0f254a9e8927e1153bb36617` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:07.808610 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f45f33b0f254a9e8927e1153bb36617)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:07.810666 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:07.872346 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f45f33b0f254a9e8927e1153bb36617'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.887643 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f45f33b0f254a9e8927e1153bb36617'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.898877 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f45f33b0f254a9e8927e1153bb36617'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.899605 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f45f33b0f254a9e8927e1153bb36617'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.909034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fa9720e-dcd8-405c-94f9-a91cac704c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f45f33b0f254a9e8927e1153bb36617'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:07.915634 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1486/2929] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:24:07 2022] DELETE /identity/v3/users/2f45f33b0f254a9e8927e1153bb36617 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:24:08.110773 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:08.113099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:08.129905 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ec912-77e4-4095-b33b-c80fba15db0d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:08.151702 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:08.201861 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:08.202550 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:08.202839 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:08.203110 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] PATH_INFO: `/v3/projects/ae25b3c0c8d5464d8779cb8bdc03cc75` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:08.204268 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae25b3c0c8d5464d8779cb8bdc03cc75)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:08.206225 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:08.256224 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae25b3c0c8d5464d8779cb8bdc03cc75', 'request_id': 'req-1217a457-1275-4a70-87cb-7015f4a766ac'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:08.263273 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae25b3c0c8d5464d8779cb8bdc03cc75', 'request_id': 'req-1217a457-1275-4a70-87cb-7015f4a766ac'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:08.270372 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae25b3c0c8d5464d8779cb8bdc03cc75', 'request_id': 'req-1217a457-1275-4a70-87cb-7015f4a766ac'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:08.285020 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] The token cache is being invalidate because project ae25b3c0c8d5464d8779cb8bdc03cc75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:24:08.285312 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:08.286003 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:08.287543 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1444/2930] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:24:08 2022] DELETE /identity/v3/projects/ae25b3c0c8d5464d8779cb8bdc03cc75 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:24:17.911926 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:18.281813 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:28.155993 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:28.156591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:28.156591 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:28.194092 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] MFA Rules not processed for user `43a6a0b75e48405ca6385ea62768d4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:28.195308 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:28.272084 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1487/2931] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:24:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:34.250719 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:34.252985 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:34.399996 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217a457-1275-4a70-87cb-7015f4a766ac admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:34.405834 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:34.406454 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:34.406734 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:34.407007 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:34.409101 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:34.423820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:34.425873 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:34.439546 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1445/2932] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:38.270115 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=37 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:44.434605 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=59 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:50.301032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:50.302567 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.401881 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f025b-d179-4161-bdff-f97dea17f488 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:50.408122 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:50.409075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.409075 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.409335 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.411174 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.424716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:50.426390 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:50.438025 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1488/2933] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:00.433282 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=41 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:02.275043 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.276754 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:02.359679 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc65e92-9efc-4b6f-a7aa-2ec386848294 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.363769 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.364390 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.364666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.365077 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.366881 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:02.380759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.380759 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.392306 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1446/2934] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:02.486263 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.498144 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df16a3-3445-4377-b83e-6679df7bb4f3 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.502499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.503108 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.503683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.504037 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.510658 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:02.528601 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.530106 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.542457 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1489/2935] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:12.386128 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=26 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:12.537606 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:54.390375 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:54.392056 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:54.498662 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b15f1-6007-4433-ae80-6d519219313a service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:54.502927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f946fc56244e179973d6f846eb7548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:54.503548 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:54.504132 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:54.504407 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:54.506478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:54.518820 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:54.520292 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:54.530862 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1447/2936] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:03.935935 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:03.938493 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:03.974615 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040c76e-6a7f-42c8-9f98-3616d9020503 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:03.978499 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:03.978892 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:03.979170 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:03.979443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:03.981176 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:04.048120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:04.049988 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:04.060651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1490/2937] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:04.526339 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=35 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:05.402099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:05.410927 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658727b-d69b-4469-ad7e-d2af0ce6497c service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:05.419807 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:05.420615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:05.420958 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:05.421234 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:05.423452 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:05.436635 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:05.438865 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:05.457263 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1448/2938] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:05.524925 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:05.534502 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fcf8a-e3ba-4878-946e-2d067fd93768 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:05.538503 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4227c0-bc8b-4edc-93e1-42d562cb33bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:05.539113 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4227c0-bc8b-4edc-93e1-42d562cb33bd service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:05.539406 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4227c0-bc8b-4edc-93e1-42d562cb33bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:05.539675 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4227c0-bc8b-4edc-93e1-42d562cb33bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:05.541593 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4d4227c0-bc8b-4edc-93e1-42d562cb33bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:05.572155 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4227c0-bc8b-4edc-93e1-42d562cb33bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:05.573627 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4227c0-bc8b-4edc-93e1-42d562cb33bd service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:05.585876 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1491/2939] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:06.806842 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:06.817949 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2888a-a13d-4b85-b468-92c7b62ccfc1 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:06.822805 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:06.823597 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:06.823991 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:06.824267 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:06.826152 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:06.832192 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bbec-839e-4b4a-8472-0f6689e9385f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:06.832726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bbec-839e-4b4a-8472-0f6689e9385f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:06.832726 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bbec-839e-4b4a-8472-0f6689e9385f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:06.843893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:06.846498 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:06.854034 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ac79bbec-839e-4b4a-8472-0f6689e9385f None None] MFA Rules not processed for user `43a6a0b75e48405ca6385ea62768d4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:06.855362 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ac79bbec-839e-4b4a-8472-0f6689e9385f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:06.860099 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1449/2940] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:06.868642 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1492/2941] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 13:27:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:27:07.317762 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:07.327976 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4bda-6c3f-4ddd-8821-527f454baae9 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:07.332710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:07.333313 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:07.333587 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:07.333965 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:07.336261 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:07.352519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:07.354180 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:07.363038 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1450/2942] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:07.873454 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bbec-839e-4b4a-8472-0f6689e9385f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:07.887844 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bbec-839e-4b4a-8472-0f6689e9385f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:07.895851 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:07.896477 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:07.897233 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:07.897505 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:07.899733 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:07.914612 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:07.917855 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:07.932584 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1493/2943] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:08.336177 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:08.351827 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2a2b3-c0d9-465c-8fbb-0a659c09184a service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:08.358006 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:08.358666 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:08.358948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:08.359216 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:08.361028 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:08.375882 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:08.378246 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:08.391356 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1451/2944] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:10.751936 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:10.765062 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e784d-1dc2-4e59-a833-eae90c11e66a service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:10.772873 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:10.773542 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:10.774074 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:10.774211 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:10.776371 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:10.893479 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:10.895904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:10.921015 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:10.925657 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1494/2945] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:14.233024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:14.234482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:14.250384 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce394dc-7b46-45d8-9984-4573592241d2 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:14.258891 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:14.259893 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:14.260397 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:14.260955 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:14.263164 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:14.276547 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:14.278664 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:14.291342 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1452/2946] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:17.920621 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:17.923040 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:17.940651 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4b28b-0769-4e71-9256-13335a8ef9d7 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:17.946118 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:17.946929 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:17.947243 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:17.947766 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] PATH_INFO: `/v3/users/abffc071dfac402087686ed80260f31d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:17.948973 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=abffc071dfac402087686ed80260f31d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:17.950986 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:18.005207 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abffc071dfac402087686ed80260f31d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.018678 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abffc071dfac402087686ed80260f31d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.027147 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abffc071dfac402087686ed80260f31d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.027815 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abffc071dfac402087686ed80260f31d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.036724 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abffc071dfac402087686ed80260f31d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.043003 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1495/2947] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:27:17 2022] DELETE /identity/v3/users/abffc071dfac402087686ed80260f31d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:27:18.232336 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:18.245531 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87a054-4b82-4dac-b5e9-f69a63c741ce service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:18.312402 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:18.313041 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:18.313318 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:18.313592 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] PATH_INFO: `/v3/projects/628aff1654fc4804bbbec0333d3537b2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:18.314745 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=628aff1654fc4804bbbec0333d3537b2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:18.316643 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:18.367478 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628aff1654fc4804bbbec0333d3537b2', 'request_id': 'req-a1ed6852-bb6b-4437-a258-f72ce13218a2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.373864 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628aff1654fc4804bbbec0333d3537b2', 'request_id': 'req-a1ed6852-bb6b-4437-a258-f72ce13218a2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.381091 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628aff1654fc4804bbbec0333d3537b2', 'request_id': 'req-a1ed6852-bb6b-4437-a258-f72ce13218a2'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.396181 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] The token cache is being invalidate because project 628aff1654fc4804bbbec0333d3537b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:27:18.396610 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.397338 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.398894 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1453/2948] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:27:18 2022] DELETE /identity/v3/projects/628aff1654fc4804bbbec0333d3537b2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:27:18.403473 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:18.405151 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:18.419842 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042412a6-24a1-493a-9202-b862de85cb9d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:18.480852 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:18.481384 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:18.481662 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:18.481961 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] PATH_INFO: `/v3/users/28b2936d3d34497b96a4dd4232afa56b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:18.483132 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b2936d3d34497b96a4dd4232afa56b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:18.485054 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:18.546245 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b2936d3d34497b96a4dd4232afa56b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.559453 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b2936d3d34497b96a4dd4232afa56b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.568220 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b2936d3d34497b96a4dd4232afa56b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.568799 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b2936d3d34497b96a4dd4232afa56b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.571077 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:18.577571 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b2936d3d34497b96a4dd4232afa56b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.584600 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1496/2949] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:27:18 2022] DELETE /identity/v3/users/28b2936d3d34497b96a4dd4232afa56b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:27:18.768360 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:18.769848 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:18.783142 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6852-bb6b-4437-a258-f72ce13218a2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:18.805722 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:18.848660 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:18.849324 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:18.849602 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:18.849908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] PATH_INFO: `/v3/projects/d9ce25b874db4eb1912b4d6def5e83f9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:18.851317 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ce25b874db4eb1912b4d6def5e83f9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:18.853567 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:18.905687 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ce25b874db4eb1912b4d6def5e83f9', 'request_id': 'req-e4b1e332-1787-4b6b-922e-00d4712022fa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.911656 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ce25b874db4eb1912b4d6def5e83f9', 'request_id': 'req-e4b1e332-1787-4b6b-922e-00d4712022fa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.918916 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ce25b874db4eb1912b4d6def5e83f9', 'request_id': 'req-e4b1e332-1787-4b6b-922e-00d4712022fa'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.937649 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] The token cache is being invalidate because project d9ce25b874db4eb1912b4d6def5e83f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:27:18.937960 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.938644 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:18.940255 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1454/2950] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:27:18 2022] DELETE /identity/v3/projects/d9ce25b874db4eb1912b4d6def5e83f9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:27:26.610219 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:26.611829 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:26.674032 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649389b-dc5a-413f-adf9-42e30c0281ee admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:26.725712 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:26.726366 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:26.726641 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:26.726902 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:26.728909 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:26.791960 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:26.793443 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:26.803227 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1497/2951] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:28.934420 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:36.798783 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=74 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:38.286615 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:38.288471 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:38.365383 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1e332-1787-4b6b-922e-00d4712022fa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:38.371300 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:38.371918 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:38.372199 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:38.372477 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:38.374482 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:38.388082 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:38.389704 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:38.400521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1455/2952] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:29:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:29:43.770904 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:43.772655 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:43.874120 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa070a19-0328-4bb4-91ab-3d3cb7bc657b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:43.883237 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd45335f7a65416a8e2535c582f0624a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:43.884049 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:43.884434 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:43.884797 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:43.886820 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:43.906439 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:43.908625 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:43.935716 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1498/2953] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:29:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:29:48.395519 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=32 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:53.917596 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=34 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:56.263148 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:56.264792 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:56.278220 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95d362-74d3-4014-a5f5-73a8b3f68776 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:56.285394 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4302574365514eb1b25bd8e62736a4c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:56.286085 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:56.286369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:56.286699 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:56.288710 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:56.364558 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:56.366044 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:56.391103 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1456/2954] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:02.982934 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:02.985552 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:03.003514 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76cd68-4b9e-41f4-9cd4-899d852ac4ab service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:03.008735 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:03.009425 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:03.009825 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:03.010164 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] PATH_INFO: `/v3/users/43a6a0b75e48405ca6385ea62768d4d9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:03.011364 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a6a0b75e48405ca6385ea62768d4d9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:03.013523 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:03.064355 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a6a0b75e48405ca6385ea62768d4d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.078702 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a6a0b75e48405ca6385ea62768d4d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.088090 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a6a0b75e48405ca6385ea62768d4d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.088849 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a6a0b75e48405ca6385ea62768d4d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.096635 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a6a0b75e48405ca6385ea62768d4d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.102683 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1499/2955] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:30:02 2022] DELETE /identity/v3/users/43a6a0b75e48405ca6385ea62768d4d9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:30:03.282640 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:03.293024 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a06c12-ac9c-435a-9dd1-a57d0e1aa7f5 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:03.355835 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:03.356386 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:03.356673 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:03.356948 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] PATH_INFO: `/v3/projects/695a571420de439581eb614b16b327aa` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:03.358133 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=695a571420de439581eb614b16b327aa)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:03.360442 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:03.410748 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695a571420de439581eb614b16b327aa', 'request_id': 'req-6d50a276-9557-4f38-b5fa-e58e4d872725'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.416623 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695a571420de439581eb614b16b327aa', 'request_id': 'req-6d50a276-9557-4f38-b5fa-e58e4d872725'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.423046 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695a571420de439581eb614b16b327aa', 'request_id': 'req-6d50a276-9557-4f38-b5fa-e58e4d872725'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.436521 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] The token cache is being invalidate because project 695a571420de439581eb614b16b327aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:30:03.436816 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.437439 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.438983 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1457/2956] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:30:03 2022] DELETE /identity/v3/projects/695a571420de439581eb614b16b327aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:30:03.443374 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:03.444866 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:03.459051 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c852-a4a7-465a-ab17-5a89ad89d3d3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:03.521622 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:03.522283 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:03.522559 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:03.522833 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] PATH_INFO: `/v3/users/0ec3f41c43df4b16b8cfcf7a026cba20` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:03.524414 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec3f41c43df4b16b8cfcf7a026cba20)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:03.526370 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:03.573821 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec3f41c43df4b16b8cfcf7a026cba20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.586141 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec3f41c43df4b16b8cfcf7a026cba20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.594847 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec3f41c43df4b16b8cfcf7a026cba20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.595538 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec3f41c43df4b16b8cfcf7a026cba20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.597757 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:03.604193 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec3f41c43df4b16b8cfcf7a026cba20'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.610940 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1500/2957] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 13:30:03 2022] DELETE /identity/v3/users/0ec3f41c43df4b16b8cfcf7a026cba20 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:30:03.792751 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:03.794369 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:03.812964 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a276-9557-4f38-b5fa-e58e4d872725 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:03.834908 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:03.873304 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1f33becd334f89a27a7404e04ed5aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6924cc44bdf49bd840f84494b6ffd4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:03.873911 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:03.874193 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:03.874469 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] PATH_INFO: `/v3/projects/4ea43de593384c98848120f29664c33b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:03.875647 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea43de593384c98848120f29664c33b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:03.877578 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:03.926083 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea43de593384c98848120f29664c33b', 'request_id': 'req-12a094bd-b74b-4939-8c47-0fe09b1f96d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.932158 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea43de593384c98848120f29664c33b', 'request_id': 'req-12a094bd-b74b-4939-8c47-0fe09b1f96d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.938504 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea43de593384c98848120f29664c33b', 'request_id': 'req-12a094bd-b74b-4939-8c47-0fe09b1f96d0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.952178 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] The token cache is being invalidate because project 4ea43de593384c98848120f29664c33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:30:03.952470 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.953123 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-12a094bd-b74b-4939-8c47-0fe09b1f96d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:30:03.954677 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1458/2958] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 13:30:03 2022] DELETE /identity/v3/projects/4ea43de593384c98848120f29664c33b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:30:13.610184 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:13.949048 dv-cinder-853315-7-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:38.307992 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.309646 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:38.478687 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa19c8c-e4a4-4ff2-bfe3-b9152f2533b5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:38.549996 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aa744-5776-4a3a-b1dd-4c1d323a21e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0c0be08934466c858ee214b06538b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d8f6b3545f548babea667c09cc5765a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:38.550526 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa744-5776-4a3a-b1dd-4c1d323a21e2 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.550809 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa744-5776-4a3a-b1dd-4c1d323a21e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.551086 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa744-5776-4a3a-b1dd-4c1d323a21e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.552867 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a7aa744-5776-4a3a-b1dd-4c1d323a21e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:38.565544 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aa744-5776-4a3a-b1dd-4c1d323a21e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:38.567165 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aa744-5776-4a3a-b1dd-4c1d323a21e2 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:38.577421 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1501/2959] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:48.573412 dv-cinder-853315-7-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}